Since you wish to visit Japan for sightseeing, you would need a proper tourist visa to enter Japan. The fact that you have a valid US F-1 visa does not change that. You can apply for the visa at any Japanese consulate in the United States.
Japan does this in a short stay format (they seem to call it a "transit" visa) as well wherein you are allowed to stay for 15 days for sightseeing or other leisurely activities. Checking at the consulate website for Atlanta reveals that your premonitions are indeed correct. On the other hand if one were to apply from India, it seems the heavy documentation is not necessary.
You'll need a valid US visa and your accompanying I-20. They have instantiated this requirement because they do not issue these visas to B1/B2 visa holders in the US. A full list is provided at the consulate website.
The answer on a previously discussed question suggests that if your trip is less than 72 hours, you can get a visa on the airport as well.