Freedom of movement is a matter of both EU and UK law, namely Directive 2004/38/EC and the Immigration (European Economic Area) Regulations 2016, which implement the directive. Also relevant is the UK law concerning giving notice to marry. These requirements can be found in the Marriage Act 1949 (as amended) and the Immigration Act 2014.
Under the spirit of the directive, and I think under the letter of the UK law, you should not need a marriage visitor visa to get married in the UK. But your case involves a few uncommon circumstances, so you might run into trouble.
The Tower Hamlets page on the question refers foreign nationals to the Home Office, and the linked gov.uk page says that you can also show an EEA family permit as evidence if your partner is not a British citizen. But that ignores the fact that partners of British citizens are eligible for EEA family permits under the Surinder Singh provision. If your partner applies for and gets an EEA family permit, you should be able to give notice. But it's possible that the guidance issued to council workers
The page also says
If you do not have a marriage visitor visa or family visa
You can still give notice of your intention to get married or form a civil partnership but the immigration authorities at the Home Office will be told.
The Home Office might:
- ask questions about you and your relationship - if this happens you may need to wait up to 70 days before getting married or forming a civil partnership
- decide not to approve your notice - if this happens you cannot get married or form a civil partnership in the UK
So the worst case is that they refer you to the Home Office. The guidance concerning the possible subsequent investigation suggests that the purpose of the investigation is to determine whether you're trying to use the marriage to circumvent immigration law, and it should be fairly easy to prove that you are not.
If I were you, I would probably think about getting an EEA family permit. If it is granted, that means that your Surinder Singh claim has been found to be valid. If it is refused, you can either challenge that or just get the marriage visitor visa.
If you don't get an EEA family permit, make sure that the immigration officer's stamp reflects your partner's admission under the EEA regulations. If the stamp says "leave to enter for six months," you should probably ask them to cancel the stamp and use a different one. In this approach, though, the immigration officer might feel compelled to go into the question of whether your claim to treatment under Surinder Singh is valid, and that might be problematic (more so if your partner requires a visa to be admitted as a visitor). You should therefore probably read up on the Surinder Singh requirements, and be prepared to show evidence at the border that you meet them.