Based on your question, I would assume you are not a native European. I would feel just as surprised if I went outside of Europe and someone told me I must not drink from a certain fountain, just as you would when you see tons of people drinking from fountains everywhere. Fountains in Europe are actually very common, and I believe there are 3 types of water you can run into, while drinking from them.
You could come across normal water, just like the one you would drink home. It has been filtered by the city's household water suppliers.
The other kind is spring water - coming from the mountains, though in most cases fountains are built at the site where a spring is rushing out locally. It is safe to drink, non-filtered natural water.
The third kind is also very common for me, since I come from Bulgaria, though not as common everywhere in Europe - mineral springs. Just as normal springs, fountains are build around the place where such a spring would emerge, but it is actual mineral-rich water you can drink for free!
The only article I could find about this in English is this rather short one.
It says that only about 30% of the mineral spring are utilized, and this would most likely be due to the rest occurring at hard to reach places. We do however have huge mineral fountains in a few cities where there are above the average by water debit mineral springs. People with multiple 10 liter (~2.62 gallon) water tubs are often sighted at these places, where they stockpile free mineral water for the week.