I'll be travelling with a first-class Germany-Benelux rail pass, and I'm planning to take an ICE train from Cologne (Germany) to Brussels (Belgium). Will I need to reserve a seat for this train? I don't know how I can find out this information.
The best way to find out whether you need a reservation on a certain train is to look up the train in a rail planner. I use the German planner for this, as it is easy to use and gives good information.
Enter the cities you want to travel and adjust date and time of day, if traveling more than a few months from the time you check, just pick a random date but try to use the same day of the week, and hit search.
In the results you will find trains with an R, a grey dot with a white R means compulsory reservation, they also have a different R for recommended reservation. Click on the orange arrow before the results and you get details.
The train you mention was in the results I got without reservation notice with it, so you can get on it without a reservation, also when you hold a rail pass.
For first class travel I might not bother with reservations but for second class long distance travel I do make them when possible. Trains do tend to fill up, also when you do not expect them to.
On that route and train (ICE) reservation is optional and costs 4 EUR with the kind of pass you have:
Here is infomation on how you can book it:
Reservations are optional on most ICE trains, notably including those Cologne–Brussels. Since you are travelling first class, there will be seats available, so there will be no need to reserve.