Theoretically, yes, you must use a Japanese passport.
According to the
Immigration Control and Refugee Recognition Act, Article 60 and 61 on Chapter 7 (English translation):
Article 60 (1)
Any Japanese national (except for crew members) who departs from Japan with the intention of proceeding to an area outside of Japan shall possess a valid passport and shall receive confirmation of departure from an immigration inspector in accordance with the procedures provided by Ordinance of the Ministry of Justice, at the port of entry or departure from which such person departs.
(2)
The Japanese national set forth in the preceding paragraph shall not depart from Japan unless he/she has received confirmation of departure from Japan.
Article 61
Any Japanese national (except for crew members) who returns to Japan from an area outside of Japan shall possess a valid passport (a document that certifies Japanese nationality if he/she is unable to possess a valid passport) and shall receive confirmation of his/her return to Japan from an immigration inspector in accordance with the procedures provided by Ordinance of the Ministry of Justice, at the port of entry or departure at which such person lands.
However, in practice you don't need it. There are many such underage Japanese who were given birth to by Japanese-foreigner couples yet born outside of Japan, but I have never heard anyone who got denied an entry/return to Japan.