Your husband's overstay breached the conditions of his leave to enter the UK, although he departed voluntarily after the initiation of removal procedures. The documentation he received would have indicated the applicable re-entry ban after voluntary departure (one year, 2 years or 5 years); it would be 10 years after enforced removal or deportation.
He certainly can apply for leave to enter the UK, and should. While a non-visa national does not normally need advance permission to enter the United Kingdom, it would be advisable to do so with such an immigration history.
Understand that his breach and removal are part of his record, permanently, and would be considered when he asks to be allowed to return to the UK. As a word of caution, if he currently barred from re-entry, it may be unwise to apply at this time, as a refusal could invoke an extension of the current ban, or worse.