I have a dual Polish  and British citizenship, but no British passport yet. If I have to leave the UK now (using my old Polish passport), will I have any problems coming back into the UK?
 Poland is an EU member (believe it or not :D)
I was in exactly the same circumstances and I managed to enter the UK without any issues.
Based on the following answer - https://travel.stackexchange.com/a/21936/42483
You can enter any EU/EEA country (which includes the UK for the next 2+ years definitely) without an ID card or passport as long as you can prove your nationality with whatever means.
The only difference is whether it takes you less than a minute to walk through immigration or quite longer (as they would need to establish that you are actually the citizen you are claiming to be).
P.S. In theory this should apply to any country in the world.