Driver's licenses issued by one U.S. state or territory are valid in all of the other states and territories. In fact, it a constitutionally-based guarantee, deriving from Article 4, Section 1. Note however that if you have a valid driving permit in France, you will be able to use that to rent as well.
You will be asked for your current address when renting, but I would not expect any problems. It is of course quite common for people to fly to other states and rent cars there for both work or pleasure. When I first moved to Maryland I often rented at an airport in Virginia with a California license, and was never once asked any questions as to why I was living in one place, licensed in another, and renting from a third.
Your tourist visa status has no bearing on the validity of your driver's license provided that you are driving yourself or companions. Running a taxi service or being paid to courier is, of course, another matter entirely.
Regarding insurance, if you have personal auto insurance at home, first check your policy to see if it will cover your rental. Second, your credit card may offer some coverage, so review their offerings carefully— rental insurance provided by credit card issuers is very limited.
Credit card insurance usually replaces CDW or LDW (collision damage waiver or loss damage waiver) which covers the cost of physical damage to the auto itself. It does not cover liability insurance (if you get sued for crashing or otherwise doing something bad with the car) or personal injury insurance (for paying medical bills in the event of injury). There is also uninsured motorist insurance, which covers you if you are in an incident and the other driver does not have the proper insurance.
The value of the various kinds of insurance are debated endlessly, so I will only say that for myself, because I do not have a personal auto insurance policy, I always get the supplemental liability coverage (it is "supplemental" if you have existing liability insurance, it becomes primary if you don't as in my case) and decline the rest. America is a litigious place.