It depends on the law of the countries and the specifics of the situation. Personally, I only know one country where carrying ID at all times is mandatory and failing to have one on your person can result in a fine (it's the Netherlands) but there are certainly others. Wikipedia has a overview of the requirements in many countries. It's mostly about the documents locals (and not visitors) must carry but as a first approximation, I would assume that if nationals have to have ID at all times, foreigners are also expected to carry a passport.
I suppose there are many countries where the police can make your life miserable (e.g. bringing you back to the police station and keeping you for several hours) but not fine you or punish you merely for not carrying ID (assuming you are legally in the country and do have an ID somewhere, only not on you). I know it's the case in France for example. Similarly, in Germany, it's mandatory to have ID but not to carry it with you (I don't know exactly what the police can do if you don't).
And then the letter of the law is only part of the problem. If some official happens to insist on seeing a real passport, arguing about legal details, in a foreign country, possibly in a language you don't speak well, might not be the best way to stay out of trouble.
Therein lies the problem with strategies like carrying a copy of your passport. It can be useful and will probably be enough in many situation but legally speaking it's probably not different from having no ID at all and you most likely don't know whether it is and are not in a good position to defend your rights. So in practice a copy of your passport might not be much help against a policeman bent on making trouble or a corrupt cop trying to extract a bribe from you.
But in the EU at least, if you are an EU citizen, carrying a national ID card should be enough for all purposes (if you have one obviously, some countries don't issue them at all). So you can indeed keep either your national ID card or your passport safe, use the other one as needed and if it gets stolen you can easily return home and renew the stolen ID using the one you stashed away.