I am not sure if the information you've received is completely correct. If you've been inside Schengen longer than 90 days, then technically the 90 days you get for free with your US passport is over and you require either a visa or an unexpired residence permit to cross national borders inside the Schengen zone. It is possible that no one will check your documents or that if they do they will not notice or care, but you should be aware that you may be taking a small risk.

The following info is country-specific and so may not apply to Sweden, but for what it's worth: In Spain, unequivocally a person in your situation requires an "Authorization to Return," which, together with an expired residence permit, is as good as a residence permit for 1 trip outside of Spain. Of 2 times that I have travelled outside Spain with one of these authorizations, I was asked to show it once coming back into the country, crossing from France by train. While flying with an unexpired residence permit, I have also been asked to show my residence permit at least once that I recall, when transiting through Switzerland. I did not volunteer it--the border control official checked my passport, saw that visa printed there was old, and asked how long I'd been in Spain.