Yes, you should be able to do that. That's precisely what the single market is about and, incidentally, the reason why the EU enacts so many seemingly petty unrelated rules (EU countries have to agree on rules regarding safety, labelling, etc. so that most products can then circulate freely between them instead of setting rules independently of each other and enforcing those rules at the border).
Information provided on the gov.uk website reflects this:
If you’re travelling from a country within the EU, you can bring any fresh foodstuffs - fruit, vegetables, meat, dairy or other animal products (eg fish, eggs and honey) - into the UK.
(This being based on EU law, French rules will be similar.)
The only exceptions to this general principle are things like firearms and drugs, for which rules haven't been harmonized. Even alcohol or tobacco can be carried between countries relatively freely, provided it is for your personal use (quantitative thresholds like those found on gov.uk are only guidelines based on the notion that very large quantities are not for personal use).