It's pretty well explained on the hotels.com website. For each night you book, they record how much that night was. Once you've hit 10 nights, you can book a hotel night for free for the average rate of those 10 nights.
If the hotel you want to book is cheaper than the average, it's completely free, but you loose the extra. If you want to book a hotel that's more expensive than the average of your 10 nights, you have to pay the difference
It's all done on excluding-tax prices, and you can't put your free night costs towards taxes. So, if your average rate is $75, and the hotel you want is $70 + $10 taxes, if you redeemed for that you'd have have pay the full $10 in taxes, and you'd "loose" the final $5 of your free night.
If you log into their website, it'll show you how many nights you've done towards your 10, and the average rate for them. You'll see something a bit like this, with all the details quite clearly explained: