I left my original answer below but for many people there is now a much better solution: GVB announced they now support contactless debit cards, Apple Pay, and Google Pay. This gives you access to the cheaper pay-as-you-go rate without the €7.5 costs for the reusable card or any hassle with refunds.
Other solutions now only make sense if you use public transportation really intensively or if you don't have a contactless debit card (or maybe a card from outside Europe and concerns about fees and currency exchange).
As you would be traveling from and back to Belgium on some international ticket you buy there, you don't need to consider using the OV-chipkaart with NS (which is a pain) and comparison is easier:
- Pay-as-you-go prices in the GVB network in Amsterdam will depend on the distance but in and around the center, you can expect to pay less than €2 per trip, I will use €1.5 for comparisons. At 3 trips per day, I would therefore reckon with €25.5 total (including the €7.5 cost of the card). For 5 trips per day, it works out to €37.5 in total.
- A 1-hour GVB ticket is €3.20. It has always been very unattractive and a bit of a tourist trap. For 3 trips per day over 4 days, it would cost you €38.4. For 5 trips a day, you would end up paying €64.
- A day ticket with the GVB is €8.50, it can save you money compared to 5 trips on the same day but would be debatable for 3 trips (that would work out to €2.8 per trip, almost double the pay-as-you-go rate but with zero hassle for credit and refunds).
- There are also multi-day tickets with per-day price going down and becoming really competitive. For 4 days, you have to pay €25.5 so €6.38 per day or between €1.28 (5 trips per day) and €2.13 (3 trips per day) per trip.
There are other tourist-oriented products, bundling the day ticket with a return trip to Schiphol, discounts on sights in the city or trips outside Amsterdam but if you have no interest in any of that, I would simply recommend buying a 2-4 day ticket.
For the scenarios you're considering, a multi-day ticket is competitive with pay-as-you-go fares, gives you the freedom to take public transportation as much as you want, avoids all the hassle of topping up your OV-chipkaart, and makes your question about refunds moot. The only way pay-as-you-go saves you money is if you end up doing 3 or fewer trips on most days or you already have an OV-chipkaart. If you're actually closer to 5 trips a day, the multi-day tickets (which you can load on a platic OV-chipkaart if you have one) win every time.
I will add however that I rarely find myself taking public transportation that frequently in Amsterdam, the city center is not that large so I will often walk and stop at different places before taking the metro on the way back rather than do multiple return trips.