In my experience, no.
I spend approximately four/five weeks in Japan each autumn. Usually in the Osaka area but sometimes I have business or leisure elsewhere in other cities. I have only ever been asked for my passport at the airports, car rental places and at hotel check in.
No one at a restaurant, bar, supermarket or corner shop has ever asked me to prove my age when buying alcohol. Indeed in the shops they look embarrassed and press the "Yes I'm over 20" button for me---probably as a courtesy assuming I cannot read it. (For comparison I am occasionally asked for proof of age in other countries. I went to a Morrison's in the north of England yesterday actually and had to produce a driver's licence to buy an ale!)
So, officially maybe you should take it, but in practise it isn't worth the risk of losing it.
In Japan, age restrictions on alcohol are not a big deal. There are vending machines selling beer everywhere.