You should wait until you have passed through customs and are fully land-side before removing the tag or discarding the receipt, as otherwise you may have some hassle. This has happened to me twice.
The first time was on a journey to Ecuador, where I wasn't thinking straight after about 22 hours of travelling and left the ticket and receipt in the pocket of the seat in front in the aircraft. An airport official of some kind was checking off everyone's luggage against their receipts. As it turned out I was the last person out of the baggage claim area, which I think helped my efforts to persuade him that I was being stupid rather than felonious.
The second time was in Mexico. I collected my backpack and joined the queue for the point where they assign you an aisle to be randomly selected or not for a bag X-ray. An official came down the line checking receipts and removing the tags from the bags. When I got near the front I saw a bureau de change, and since I had no pesos at all I left the line to change a small amount. After I rejoined the line, I was checked again by someone else, and asked why my backpack had no tag. Of course, the person who had done the first check was now nowhere to be seen.
In both cases I managed to straighten things up by explaining what had happened, but that kind of interaction is unnecessary stress when you least want it and could go much worse.
On the other hand, once you're land-side there's a reasonable argument that you should remove and conceal the tag. You're almost always going to be an obvious tourist, but if you can at least not be obviously a tourist who has literally just arrived in the country then that can help to reduce the "scam me / rob me" vibe.
Depending on your level of concern about identity theft you may wish to keep the tags until you can shred them to reduce their usefulness to "dumpster divers".