In Tokyo, the area between Yotsuya station and Shinjuku park is exactly like that (a friend of mine lived there for a year) and very central (just a few hundred meters from the imperial palace). I'm not sure there's a place to stay short-term in that area though.
Using Google Maps, it should be easy to find such areas in any Japanese city by looking for many small houses built close together near the center. However, do you speak Japanese? If not, it will be very difficult to really "experience life as a local" in any meaningful way.