I worked for Bell Canada for a while and this is a question that came up a lot, mostly with people vacationing overseas. I don't know much about Orange or French plans but here's some things to consider:
How does your current plan compare to other plans on the market? If you have an older plan, it might be cheaper than any plan you would be able to get if you cancelled.
What's the bare minimum you could pay for your current plan? You might have add-ons that you can remove or reduce while you're away while keeping your current account, ex. International calling, data usage.
Cancellation/payout fees. I don't know what it's like in France but in Canada, if you try to leave a contract, you have to pay off the balance on the phone. This can be several hundred dollars, way more than the cost of your bill for four months.
One thing to look into is if your phone can be used in other countries if it's unlocked. Unlocking a (Canadian) phone is usually between 35-50CAD, and a SIM card is 5 or 10$ depending on the carrier. Pay-as-you-go plans are quite cheap, but you have to buy a phone outright (min 100CAD) or already own one. If you can do that, then you might be able to save enough to justify not cancelling your current plan.
Also, getting a phone set up is pretty quick. If you're going with a prepaid plan, you'd probably be looking at about half an hour.