The Royal Palace is easily reachable by foot from the train station - just follow Karl Johans gate ("gate" is Norwegian for "street", not an actual gate).
Aker Brygge is a nice place to watch the harbour.
The Norwegian National Opera and Ballet is also nearby if you're interested.
Akershus Fortress from 1290 and the Viking Ship Museum should fit the "ancient architecture" ticket mentioned above nicely.