According to the Japanese Ministry of Land, Infrastructure and Tourism:
- The tour conductor should "Ensure compliance with infection prevention measures throughout the tour."
- The tour conductor should "Keep a record of the tour participants' activities."
- The traveler should "follow the instructions given by the tour conductor."
- "Failure to comply with the above may result in you not being allowed to participate in or continue on the tour.
You may also be required to immediately return to your home country during the tour"
So you Legally have to follow the tour conductor instructions. And you can't visit on your own if the tour conductor is against you doing activities alone.
The question is if there is a tour conductor that would not be against you visiting and touring japan on your own agenda.
It seems very unlikely as that would go against the first two items of the bullet point since he would not be monitoring you but rather just taking your word that you will respect preventive measures and report all activities you do truthfully. And they would not take risk just for you, they sold a guided tour and that's the extent of what they have to provide.