I wouldn't expect much temperature consistency from your average we-also-have-a-pool hotel pool. If you simply have a hotel which simply includes a pool, you'd probably need to contact the hotel in advance to figure out whether they place any importance on its temperate and what it's set to.
The presence of a “swim-up bar” is a good sign that it’s a pool heated to a comfortable temperature.
If you're specifically looking for a warm-to-hot bath to relax in, you'll probably want to look for places centred around this very idea, or at least hotels which offer specifically hot water pools. These may have different terms in different parts of the world.
In Europe, you'd be looking for a thermal bath (de: Therme, fr: thermalisme, es: termalismo and similar variants). These usually have swimmable warm pools and separate hot pools, in addition to saunas. You can find thermal baths to just spend a few hours to a day, and hotels build around thermal baths.
In Arabic countries (and as exports from there), you'll find hammam or Turkish baths with similar roots.
The general international term is spa. Depending on the country, this may or may not focus more on massages or saunas and more or less swimming, but you'll surely find warm water there in some form. If your hotel specifically advertises a spa, you can be fairly sure it has a "loungeable" pool. More or less random example: https://soranohotel.com/en/spa/.
In some countries there may be specific local variations like onsen, which is its own cultural thing to the degree that you specifically go there to experience it. You can again find onsen to dip into for an hour, and guest houses or entire towns designed around the onsen to spend a few days at.
Internationally similar places are generically known as hot springs or hot spring resorts, which may or may not have local cultural significance the way onsen have in Japan.