While this question is specific to Japan, there really is one answer that fits all.
If you are even slightly in doubt, don't.
Even if you are following the law, to the letter and have measured your knife to make sure it complies, you run the risk of:
A custom's agent having a bad day. If he says "This isn't allowed" and you argue with him, things could go badly for you. You might not be arrested or detained, but you might also not get let into the country.
The person who answers your question might be incorrect or have outdated information. This applies even when asking someone who works there. People do make mistakes. Don't make someone else's mistake, especially when a weapon is involved.
Subjecting yourself to additional scrutiny over an object that you could easily purchase once there.
When it comes to immigration, customs, laws about what you can and can't bring in or out of the country, etc - always err on the side of caution. I visit several new countries in Asia every year, when I pack - I pack to make sure that nothing on or about me attracts the attention of anyone who has the authority to make my day go rather badly. I don't even bring my Swiss Army knife key chain, for instance, because you really never know the mood or disposition of the person who has the job of letting you into the destination country.
If you really want to bring it with you, ship it to someone there who can keep it for you, then ship it home before you leave. Either way sounds like a bit of a gamble to take with something that obviously means a lot to you.