Not unless you are trying to rent one during a busy holiday
Generally as long as the company has cars to rent, you can rent them without a reservation. Stock usually runs low during holidays, long weekends and local festival days and as such this is a time where you should reserve a car.
Renting in the greater Tokyo area is usually painless but does require you have either a Japanese Drivers licence for the type of car you intend to drive (ie. Automatic licence for Automatic, Manual licence for Manual) or an International Drivers Licence. Some companies also expect you to get a more comprehensive insurance policy if you are on an IDL.
Also, reserving a car guarantees you get whatever car you have chosen (rather than just whatever is available on the day), as well as being able to chose certain plans and get online discounts with some companies.
Both myself foreigner on a Japanese Licence, a foreign friend on an IDL and a Japanese friend have rented on the spot before and this is unlikely to change in the lead up to the 2020 Olympics.