I tried the standard tools like Kayak or Momondo, but there the cheapest option is always a bed in a dormitory. In particular I was searching for Amsterdam and the first few pages of results (sorted by price ascending) were full of dorms, making the tools useless in practice. How can I efficiently filter on the fact that I want a private room (for whatever number of people I enter)?
On several of the hostel booking sites you can filter by room type, but this doesn't really filter the dorms away.
On Hostelbookers you can do a search and then sort by 'Private room price (low to high)' which gives you the lowest private room prices first and you can ignore the shared prices which are still listed.
Almost every hotel and hostel booking site allows you to filter/sort to identify private rooms.
Of course, not every hostel has private rooms, but most modern ones these days offer that option, and certainly hotels will allow this.
You can use Hostelz.com to search by either dorm beds or private rooms (in fact, you have to choose one or the other, it always searches dorm beds and private rooms separately). It's also possible to filter by room/bed type on Hostelworld (I think that feature was added recently), but it still isn't possible on some of the other hostel booking websites. Hostelz.com searches all of the hostel booking sites at once (similar to what sites like Kayak let you do for hotels and airfare, but for hostels), so it will show you the lowest price for a private room or dorm bed from across all of the booking sites. To order the results by price, look for "Order By:" on the left side and choose "Price".
There is definitely a big difference between wanting a private room and dorm room. I'm not sure why all of the hostel sites don't make that differentiation when searching for availability and prices.
Disclaimer: Just to be clear, I am posting this as a representative of Hostelz.com, so my opinions are probably at least a little biased towards the particular way our site does things.
Booking.com has this functionality. Search for hotels in a given city, then sort by price:
Booking.com will then offer you to exclude shared rooms: