Thanks a lot for the answers. In the end, it worked out for us.
The embassy could not issue a Travel Document for Return to Japan; they only issue them in case the passport has been lost or stolen, and a corresponding police report would be necessary. Instead all they could do was issue a new ordinary passport, which under emergency circumstances can be done in 24 hours. For this, one normally needs an original of a Koseki touhon, but a copy will suffice as long as one promises to bring the original version (exactly the same version that was copied) as soon as possible to the embassy.
This was not an option for us, so we contacted the 外国人在留総合インフォメーションセンター ( https://www.moj.go.jp/isa/consultation/center/index.html ).
They said that with an original of a Koseki touhon, it is possible to enter Japan even though the Japanese passport is expired. After contacting the KIX airport, we were told that someone already in Japan (a parent, etc) can bring the koseki touhon to us inside the airport before we go through immigration. This is how we solved it.
We were also advised to keep a copy of the koseki touhon with us when traveling, although we did not need to show this.
For the PCR test result, we were advised to put the expired Japanese passport number, and not the passport number of the valid EU passport.
Finnair never commented on the expired passport and simply checked us in. We also called them beforehand and were told that as long as the child has a valid travel document (the EU passport) and it is apparent that the child is a Japanese citizen, they will not deny boarding.