When I entered with my Polish passport through Saint-Georges, I was told by the border police I'm not supposed to get any stamp cause we're in France, the same was told when leaving in Saint-Laurent-du-Maroni. But then I was stopped twice inside the country by the gandarmerie, twice they complained that I don't have the stamp and I should have gotten one. In both cases they took my passport for good 15 minutes, made some calls, asked a lot of questions, and finally let me go, but I could see they weren't sure what to do. I talked with a Latvian guy in my hotel and he got a stamp when he arrived in Cayenne.
As far as I can tell from Article 355(1) of the consolidated TEU/TFEU, French Guiana is part of the EU:
- The provisions of the Treaties shall apply to Guadeloupe, French Guiana, Martinique, Réunion, Saint-Barthélemy, Saint-Martin, the Azores, Madeira and the Canary Islands in accordance with Article 349.
Free movement therefore applies, and you could have entered with your ID card instead of a passport. If that is correct, then the gendarmerie needs better training, since one element of free movement is a policy of not stamping passports of EU citizens when they cross EU borders.