I've just travelled on this train in the opposite direction, all the way from Vancouver to Toronto, about a month ago. This is about the Via Rail trip, which I assume you're taking instead of the tourist-focused Rocky Mountaineer. I don't know much at all about the latter train, so this is about the Via Rail one.
Long-distance passenger trains in North America are increasingly hard to run on time, as they mostly traverse railroads owned by freight companies, who naturally prioritise their customers' freight (especially for things like perishable goods or goods that are hard to stockpile where the customer will have to halt production if they run out) over these pesky passenger trains, which they would rather do without being on their railroads. Making this even worse is the fact that even when they're not time-sensitive, freight trains now are often longer than the sidings (passing points) on single-track lines, meaning that it's the passenger train that has to wait in the siding for the freight train to pass rather than vice versa, as the freight train won't fit.
The schedule was recently lengthened for this reason, with lots of padding added. As a result of this the end-to-end journey is now usually approximately on time, though you should still plan for the event of delays in the order of a few hours. I imagine the difference in outward and return journeys is simply due to differences in where the padding was placed.
There were a reasonable number of people on the train when I went in late November. The train in summer can reportedly be very busy, but in the winter is quieter. There are still enough people on the train to make the journey interesting, though! The journey is quite scenic whatever the time of year, but obviously during the winter the days are substantially shorter so beware that you might not see the bits you were hoping to see in daylight, especially when combined with the uncertainty of delays or running early!
The train is broadly split into three classes — Economy (seated), Sleeper Plus (various options for sleeping accommodation of different levels of comfort), and Prestige. Economy class gets you a seat and is not generally too expensive, but you have to buy food and drink from the Economy dome car, which is generally cheap microwaved fare.
Sleeper Plus class gets you a bed (either in a private room ("cabin") or in a corridor with a thick privacy curtain ("berth"), depending on precisely which type you book) which converts into a seat for daytime travel, and access to the dining car for proper sit-down meals cooked by a chef which are included in your ticket price. Those unfamiliar with dining on trains might be surprised to learn that you would usually be seated with other people (the tables are for four, and there's always limited dining space, so it wouldn't make sense to give every group their own private table), but really this is quite a good experience as it's the perfect opportunity to strike up conversation with perfect strangers and learn what brought them there. The food in the restaurant, in my opinion, is great; I doubt many people would have any complaints with it. You also get access to a dome car for Sleeper Plus passengers only, as well as the Park Car sometimes (see below), which is a dome car right at the back of the train with the best views, including views out the back window along the line, and a bar serving snacks and drinks.
Personally, I travelled in Sleeper Plus class, in a Cabin for One (I was on my own on this particular trip). The main reason I chose this over the Berth option (which is the bed in a corridor with a thick privacy curtain) is that there is no power outlet with the berths. As I was doing the full four night journey this was quite important for me!
Prestige Class gets you by far the most luxurious accommodation, as well as private talks by a host on any subject that interests you, and perks like access to the best seats in the Park Car (and sometimes, exclusive access to said Park Car).
The train being busier in summer means there are more organised activities etc., but on the flipside it also means that some parts of the train are more restricted to premium class passengers. For instance, the Park Car is restricted to Prestige class passengers only for much of the daytime in the summer, whereas anyone in Sleeper Plus class can visit it at any time in the winter.
As for what you generally do while on the trains? Sit, chat, admire the view, read, play cards, watch films (with headphones of course), eat, drink, and just generally enjoy having a long period of time where you can truly relax. It's not often people get this in the modern world, and I found it to be very enjoyable myself!
I will say right now that I'm biased, as I personally love trains, and live in a place where train travel is generally considered the best way to get around the country. But The Canadian really is great, and even for a relatively short journey like this, I think it's a complete no-brainer and the best way to travel.