(I think this is more of a question for https://law.stackexchange.com/ than travel but I shall have a go at it.)
They are not merely in violation of a rule, they are in violation of the law and potentially committing a criminal offence.
Entry to the UK is tightly regulated by the Immigration Act 1971. The Act has been amended by many later enactments and statutory instruments, but helpfully all the relevant provisions remain within the 1971 Act (and the orders made under it). Probably it is wise to read the whole Act and all recent statutory instruments made under it. But the key points are---
3. General provisions for regulation and control.
(1) Except as otherwise provided by or under this Act, where a person is not a British citizen—
(a) he shall not enter the United Kingdom unless given leave to do so in accordance with the provisions of, or made under, this Act;
The Act is very clear, you must only enter the UK within the provisions of the Act. The argument, "Of course it's easy to say that since it's UKVI's fault the person has no liability", I would submit, is not easy at all to say in light of that law. For one, it is not a question of liability. The foreign citizen either has leave to enter or he does not. Who is liable is unlikely to be important in considering the legality of his removal; and it is not clear that Parliament intended the entry clearance to work on that kind of basis. If the immigration officer allows him admission in error or neglect, that is only acceptable if the Act (or orders made under it) directly provide that it is so.
[Continuing with the theme of 'fault', the House of Lords ruled in Khan (1977) that a person could be removed from the UK on the basis of another's deception in securing her entry clearance, of which she was unaware. Therefore I would surmise that the point is not to place blame on one person or another, but rather to find objectively if a person is permitted to enter, and whether she obtained proper clearance to do so. If the clearance is not valid, then, unless extenuating circumstances apply, the person must be removed. http://www.refworld.org/docid/3ae6b6ca0.html ]
Moreover, "given leave" has a clear meaning. You are not, I think it is fair to say, given leave by an omission, by walking around the back of the immigration desk. Being "given leave" must be a purposeful act. It could, I suppose, be in error by a man (with authority to admit you) waving you through.
Later in the enactment an offence of illegal entry is created.
24. Illegal entry and similar offences.
(1) A person who is not a British citizen shall be guilty of an offence punishable on summary conviction with a fine of not more than level 5 on the standard scale or with imprisonment for not more than six months, or with both, in any of the following cases:—
(a) if contrary to this Act he knowingly enters the United Kingdom in breach of a deportation order or without leave;
The word "knowingly" is a good argument here, can you be okay with just 'accidentally' walking out of the airport? Or 'accidentally' booking the right combination of trains and forgetting to get off in Paris? I think a judge would be very sceptical to such an argument. A traveller knows, or would reasonably be expected to know, that he must talk to immigration and receive a stamp.
Moreover the Act deliberately shifts the burden of proof onto the traveller, who must prove that he was given leave to be admitted (providing it is shown he entered in the last six months).
(4) In proceedings for an offence against subsection (1)(a) above of entering the United Kingdom without leave,—
(b) proof that a person had leave to enter the United Kingdom shall lie on the defence if, but only if, he is shown to have entered within six months before the date when the proceedings were commenced.
I suggest that is rather difficult to prove you were legally admitted if there is no stamp and no record of your entry except for a train ticket or a record on the airline reservation system.
Nonetheless the two points are distinct. The entry may be unauthorised ("without leave") without the criminal offence being necessarily being committed.
Certainly the person can be removed from the United Kingdom, at the expense of his inbound carrier if it is known. Carriers are strictly liable for transporting unlawful immigrants even if they do not know the person will be denied entry. Here is the official guidance: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/270022/chapter47.pdf
Turning to case law, an early example of the deference shown by the Courts to the government is in Khawaja v Secretary of State (http://www.bailii.org/uk/cases/UKHL/1983/8.html). The facts are a bit different to what you describe (there was deliberate deception), but one thing that stands out is,
if the immigration authority has reasonable grounds for believing that a person is an illegal entrant, the decision to remove him and to detain him until he is removed is for the authority. It is not subject to review by the courts, save to the limited extent recognised by what has come to be called "the Wednesbury principle".
Although the Courts move back and forth on this issue, the Immigration Act gives the Secretary of State wide powers to remove illegal entrants (or even legal entrants) without the possibility of judicial intervention, save in some Human Rights cases and other cases contrary to general public policy (e.g., person could be tortured or face death on return).
It is quite difficult to find specific examples of people accidentally entering the UK that were dealt with by a recorded judicial proceeding on bailii.org. So I shall have to leave my answer here for someone else to complete.
Nor am I aware of a means of "normalising" one's status. One can seek asylum of course, but it is probably better to exit quietly and later seek re-entry.