It's possible that the answer will vary depending on the specific embassy/consulate, but to use the US Consulates in Australia as an example...
No, you can not. Even as a US citizen you are required to make an appointment to visit the consulate for non-emergency services.
As stated on the US Consulates Australian webpage :
To be allowed entry into the U.S. Consulates for routine
(non-emergency) services, you must make an appointment online.
There is seemingly an exception here for emergency services, however I doubt they would classify "I just wanna have a look around!" as an "emergency".
The appointment system requires you to enter the purpose of your visit, and once again "just lookin'" isn't on the list.
The equivalent page for Japan has different text, but still states that you must "schedule an appointment"
It's worth keeping in mind that US consulates have an extremely high level of security, and the answer for other countries consulates/embassy will probably be different. For example, to access a US Consulate in Australia you need to have an appointment, have your bags X-rays, go through a metal detector, and leave all electronics at the security checkpoint. By contrast, to access an Australia consulate in the US you need to ring the doorbell (or at least, that's been my experience at the Australian Consulate in San Francisco!)