I would love to drive through Brazil and Suriname starting from French Guiana. However, renting a car to do this proved to be impossible. There are ferries between French Guiana and Suriname and a bridge between French Guiana and Brazil. This implies French registered car do cross the border to Brazil and Suriname. If I am not a resident of France, would I be able to buy a second hand car, insure it and cross the border to Brazil and or Suriname?
You ask many questions, and the answers are several.
Yes, you can buy and own a French car while not being a resident.
You can insure it either through your own company in whatever country you are in (for a short or long term period) or through a French company. The price might be high, since you might not offer the highest guarantee for the French company. Insurance companies have complex models to calculate the amount to pay, and foreign owners of cars in French Guyana is fairly complex.
Crossing the borders is a different topic. 3.1. Regulation from Brazil or Surinam might prevent you from crossing the borders freely with your car; you might need special waiver etc. 3.2. This adds to the problem of insurance. Usually French insurance companies insure for the French territory (and the European Union). It might be difficult to have an international car insurance covering Brazil and Surinam (or more costly).
Of course, it is mandatory to have insurance in France (and French Guyana), it might not be the case in Brazil or Surinam). If you have a very beat up car, you might not care that much.