A Canadian Citizen with an Indefinite Leave to Remain in the UK, I am shortly to get naturalized as a British Citizen. However arranging documents to get a Passport will take time. As a British Citizen can I travel outside the country and return on my Canadian Passport. I will have a British Citizen Certificate.
Despite what the government says at the page linked in the previous answer, you cannot be denied entry to the UK if you can show that you are a British citizen. Since Canadian passport holders can fly to the UK without a visa, you can leave the UK and return with your Canadian passport along with proof of your UK citizenship.
See a related answer in which a Canadian who is also a British subject with right of abode reports that she answers the question about not using her UK passport by saying
that I've lived in Canada since 1969 and that it got to be too much effort to renew my British passport. Then they let me in.
In your case, if they ask, you could say that you haven't yet had time to get the passport.
I would also note that the statement "you cannot enter the UK using your BRP or certificate of British citizenship" makes no representation about whether you can enter the UK with a certificate of British citizenship and a valid foreign passport.
No, you cannot use the certificate to enter the UK, once you attain citizenship (congrats).
Once you have a British passport you must use this to enter the UK.
If you do not want a British passport you can apply for a certificate of entitlement instead.
You cannot enter the UK using your BRP or certificate of British citizenship.
After you get your certificate
You must send your biometric residence permit (BRP) back to the Home Office within 5 working days of getting your certificate of British citizenship.
Cut your BRP into 4 pieces and put it in a windowless envelope.
Include a note saying you’re returning your permit because you’ve become a citizen. Include your name, date of birth and the document number (found on the front of the card) in the note.
Naturalisation BRP Returns PO Box 195 Bristol BS20 1BT
You’ll be fined up to £1,000 if you do not return your permit within 5 working days.
Note: The information found on the gov.UK site has been transposed to first respond to your query, then add the advisory regarding your biometric residence permit (BRP).