This answer is for all others who are considering the same trip:
Kutna Hora is one of the several tours ran by Prague Tourist Agencies, so not exactly without tourists. Many busloads are coming to Ossuary, St.Barbora Catedral and having lunch there before returning back to Prague. Taking a bus from Prague and walking around on your own will be much more enjoyable and just a small fraction of what the travel agencies charge.
If you really want to go, take a bus which is leaving from Cerny Most terminal to Kutna Hora. We enjoyed the bus trip very much as it goes through small towns and villages, stopping in each of them. You can really see how ordinary Czechs live. The bus trip is also cheaper than the train. To get to Ossuary, you take a very short ride on Kutna Hora local bus to Sedlec. Bus stops close to a small chapel (you will see parked busses), Ossuary is very small and often packed with tourist groups. However, the city of Kutna Hora is nice and worth your time. Avoid restaurant U Sv. Barbory where I had a worse lunch in my life.