My answer will be a bit more general and won't limit to Norway/Swedish only.
It may or may not be rude, depending on what you want to achieve and how you approach this.
First let's point out one thing. The goal of any communication is to efficiently exchange messages between participants. Whatever method works, if you've managed to pass the message your communication was successful. Having this in mind it's absolutely fine to use any means of communication you have at hand, especially finding any mutual language is absolutely OK as long as both/all parties agree to use it.
If I were in your position, I would start with Norwegian, just to show courtesy. Then once I felt no longer comfortable (e.g. I felt I am unlikely to understand an answer to my question) I would ask if we can switch to some other language, suggesting English or Swedish. Sometimes (if I'm sure the other party knows English, e.g. at work or at a hotel) I would just switch to English apologising that my Norwegian reaches only that far. In Norway and other countries with a well developed English learning system it is very likely you'll never get to the point where you can suggest changing the language. It'll be the local person who'll immediately recognise you have difficulty speaking their language and offer you a language switch, probably suggesting English. There you are, continue in English and if there are some problems with that as well, you can suggest you know also Swedish (and whatever other languages, especially somehow related to the local language, so German/Dutch are good options too) if that can help the other participant.
Last few resorts, when finding a common working language fails is to:
- use a mix of languages, ask for clarifications whenever needed
- add hands to your conversation; it's amazing how much can you explain non-verbally when in a real need
- use Google Translate (as you could see in Russia during recent Football World Cup); it can really work and does more than decent work, especially if one of the parties can use proper English
- find an interpreter
As for using Swedish rather than Norwegian to show courtesy only when you know your communication is going to be better in English, I would be very cautious. You aren't actually using the local language anyway so you might be involuntarily suggesting you don't care about the difference (so actually the local language). While it should no longer play its role, remember the history between Norway and Sweden wasn't always as good as it is now. Norway gained independence from Sweden only in 1905. So I would not go that path.
Few examples from my own experience, some related to Norway specifically, some in other setups, that might give a better understanding.
- I was trice in Norway. When I was 6, 13 and 26. As a kid I knew neither English nor Norwegian. I didn't have to communicate in any of those languages in general as I have Polish speaking family there but I remember I was happy to learn at least few words from my aunt's neighbour. When I was 13 I already knew very basic English. I was able to communicate with my Norwegian colleague (about the same age) using a mix of English, Polish, Norwegian, hand-waving and pointing things. We understood each other perfectly. I also had a funny encounter with a local who happened to ask me something in the train station. I managed to say (with a mistake) the only sentence I knew: "I for stor ikke" (should be "Jeg for stor ikke"), which means "I don't understand". The guy thought it over for a brief moment and immediately switched to English (and I was so happy to understand him and be able to answer that I remember it to this day ;-) ). During my last trip I was using English entirely and it didn't fail me once.
- I travel a bit and usually I learn at least basic greetings and things like "Thank you", "Yes", "No" etc. Sometimes I push a bit more, and for example in Lithuania I was able to order food in Lithuanian. Yet usually that's all I can say in a local language and switching to English the way I described worked just fine with just few notable exceptions I'll mention later
- I've been several times in Czech Republic and Slovakia, two countries which were separated even more recently than Norway and Sweden. These were mostly business meetings and most of the discussion was done in English. Yet sometimes we were using a mix of English/Polish/Czech/Slovakian (depending who was in a meeting) as those Slavic languages are so close to each other we could mostly understand each other. Only later I found out that whenever the guys spoke between themselves they always used their native languages so Czechs were speaking Czech and Slovaks - Slovak. It was common for them. Yet I believe if I tried to use poor Slovak against Czech and especially poor Czech against Slovak it wouldn't be too welcome, especially when I am able to communicate in English.
- In Lithuania the knowledge of Polish is common. Yet some younger people reject using it due to some bad history between our countries. Usually English worked just fine. Yet I had a situation where I had a problem as no-one could speak even at mediocre level (at the airport!) and it turned out the seller was perfectly happy to speak Polish (by sheer accident, I just referred in Polish to kids accompanying me and the seller happily said in Polish "Oh, I can speak Polish"). Yet I know enforcing Polish would be plain rude unless clearly accepted/offered by the other person.
- In Italy English failed miserably. In several situations I was able to say more in Italian (and I really don't know that language) than the guys could understand in English. At one case I've asked as politely as I could "Parli inglese" (Do you speak English) and the guy whom I asked just jumped into the car and drove away (he just stopped a moment before at the fuel station so I don't think he was going to depart that soon).