The other answers are excellent, and correct. I wanted to share a few extra ideas because you specifically said:
Do note that I am very shy and submissive in public.
I also am a shy person that's, for various reasons, done a lot of travelling on my own. What I always say to myself is:
You'll never see any of these people ever again.
And that gives me (in my mind) permission to be someone else, to tell people no, to argue that I'm due the seat I paid for, or to sing karaoke, or whatever. Particularly if they're not asking nicely. Any embarrassment I feel is going to disappear at the exit of the airport, at the latest. Travelling alone is a great chance to try things you normally wouldn't do -- and, who knows, you may feel comfortable doing them when you're back home as well. Everyone else is right, it's completely your right to refuse, but if that's really against your nature just pretend to be someone else for the flight. Don't go overboard, but you can be someone that's pleasant but firm and fair -- and who doesn't worry about it,
That's one side of it, but from experience somedays you just can't, on those days I use avoidance tactics that you can employ too. Put headphones in when you sit down (there doesn't have anything playing), close your eyes and relax. Only the rudest of the rude people will target you to ask to change seats in that situation. And you'll avoid any small talk with the people next to you if you're not feeling up to it.
But, above all, however you do it, do what everyone else said, and just say no.
No
, on the other hand, is."very good I assure, come come I switch"
and proceeds to pick up your backpack. In that particular case I did call the flight attendant, and deliver unto this man some choice words. I live in India, a land of many wonders. Unfortunately, basic courtesy is not common.