IKeelYou's answer is excellent for the general case, but I'd like to add some specifics, since you mention this is in China.
This guy was trying to sell you transportation service. This is a very popular way for people with cars to make money, because just getting tags (i.e. current stickers, registration, license plates, etc.) is both expensive and difficult, easily costing ten to fifty thousand yuan, plus there are a limited number available. That's just tags, mind, so it's on top of the cost of the actual vehicle. Having a car in China is both very expensive and a significant status symbol. Technically, he's breaking the law by operating as an unlicensed taxi, but the reality in China is that nobody cares as long as nothing egregiously illegal happens.
Touts in China are aggressive, especially for something as completely fungible as a taxi ride, and especially to foreigners (I'm guessing you aren't ethnically Chinese or are somehow otherwise obviously not a local). Following a likely person (e.g. foreign, relatively wealthy) around to try to convince them to buy is extremely common. The idea is to pester you until you give in and pay for a ride just to shut them up. The best way to deal with them is to refuse to acknowledge them at all. Don't say anything to them, don't look at them, just act as if they don't exist and walk quickly past. If they are really insistent despite this, turn around and say "No!" loudly and firmly. Don't be afraid to put a bit of edge in your tone and assume an angry countenance; this will make it crystal clear that you really mean no. As opposed to being polite and friendly ("oh, that sounds nice, but no thanks"), which will be taken as a signal that they should work harder to convince you.
Unlicensed taxis are quite common in China; there's nothing unseemly or shady about taking one, though they are less popular than official taxis with the locals, what with being technically illegal and all. However, it's definitely better to take a regular taxi if you're a tourist, have language difficulties, or are generally unfamiliar with the area. Not because they'll kidnap you or some such thing, but because you'll get taken for a ride metaphorically as well as literally; i.e. end up paying a hefty "tourist tax". Official cabs have meters and posted signs about the cost, so you know how much it should be (though that's not to say they don't have tricks they can pull to up the price), while prices in an unlicensed taxi are always negotiable (and usually higher than an official cab). Taking a proper cab also means you know the driver has a license, is more likely to know how to get where you want to go, and has at least some skill operating the vehicle. On the other hand, the illegal taxis are much easier to hail and will operate at hours when regular taxis won't. So if you really want a ride right at that moment...