This is pretty typical for Near/Far East/Asia, and it gets more pronounced the farther east, and south, you go. Also, the poorer the place is overall, the more pronounced.
Your question "How do you deal in a polite way" already shows the main issue. That is, pardon me for being blunt, a facepalm statement.
You only have two options (well, three actually):
- You just pay.
- You are prepared. And no, politeness is not part of your preparations.
- You stay home.
Being prepared means you have an approximate idea of where you are and what -- approximately -- it will cost you. Ask at the hotel if unsure, they'll know. With cell phones, GPS, and Google Maps, this is nowadays a trifle, anyway. But it was easy and doable 20 years ago with a map on paper, too -- no problem. It's not like you drop by parachute at a haphazard random location in an unknown city. You have a rough idea about where you're going ahead of time.
You have the adequate amount of money ready, and not in your wallet. If necessary, get a couple of small bills in your hotel for that very purpose. You must expect not to get change (it's a fair assumption that any bill that ever leaves your hand will never be seen again), so large bills necessarily mean you will pay more.
Before going, make sure the driver says that they have understood where to go, and that a price which you are willing to pay (probably 50% more than you should pay anyway) is agreed on. Show the driver that money and be explicit that this, and nothing else, is what he will get, no more. If the driver doesn't want to agree, walk away.
Do not, ever, ask or leave any doubt about what you're going to do or pay (you may ask for the price, but if that is more than you are willing to pay, which is pretty certain, it must be 100% clear what you will be paying, and that you will not make a compromise). Do not, ever, show that you are uncertain.
Do not get your wallet out, do not ever let them look inside or even see it. You do not even have a wallet. You do not have any other money. This is the only money they will ever see, or get. Maybe, that is. If they do as you say.
Inevitably, they will start to haggle. Do not engage in this. It is "Yes or Yes", not anything different. Inevitably, they will haggle again after arriving at the destination. Do not engage in this. Give them the money you agreed on, turn around and go.
You are not polite, do not even attempt to be. You need to realize that you are not at home, and you are not among friends. The social rules of interaction are not what you are used to.
If you are trying to be polite, you are weak, and there's only two kinds of people in the world: those whom you exploit, and those that you respect. There's those who (figuratively) slap you in the face, they're the ones you respect, you are grateful to them for slapping your face. And there's those who you steal from. Because, well, they're weak. They're victims.
This is harsh, and it may not be agreeable with your normal attitude (it took me a long time to learn this, too), but it is reality. If you cannot adapt to this attitude, then you either shouldn't travel, or you should just accept that you will be ripped off, and pay. Because either way, you're going to pay. Because you are a victim, you just haven't understood that yet.
Note that the same thing can happen to you in Europe as well (doesn't have to be Asia, only just it's much more common there). Happened to me, much to my surprise, in Bordeaux 6 or 7 years ago.