Norway has one train service that covers the whole country, but you have to be aware that although you can technically get to "anywhere", the rail service for southern Norway is (almost always) connected through the capital (Oslo). In practice this means that two destinations that are fairly close together (such as Bergen/Stavanger or Gjøvik Lillehammer) require a stopover in Oslo to switch trains and go back. This makes the trip way longer that it has to be (way longer!)
But there are also busses and ferries covering more direct routes. These are often covered by more local companies, so what options you have and how to book them is highly dependent on where you are actually planning on going. It is also worth mentioning that Norway is very long. If you want to travel from the south to the north it is way faster (and potentially just as cheap) to book a flight as it is to take the train or a bus.
To answer a general question with a general answer: Yes it is possible to get to most (if not all) places in Norway using public transport. But you have to know which companies are providing transport where you are going, which can vary from place to place. Services like Rome2Rio can be a good start to discover options, although in my experience it is not always 100 % reliable and might suggest some weird connections that makes no sense.
Or, 2023 update, try Google maps and alike, as public transport in those is getting more reliable.
