It happens, not frequently, but it does.  Here's an example:
[JFK allowed passengers arriving on international flight to exit without going through Customs][1]

The mistake is as simple as a door being in the wrong position to direct arriving passengers.  That's it.  I've never seen anyone actually directing the flow.

Basically, the arriving passengers must be identified and processed.  This is not a free pass to enter the country.  Also, the terminal is not cleared so any domestic passengers might not really notice anything other that airline and airport personnel running around.

If passengers can be identified before leaving the airport, they are merely escorted back through the arrivals hall.

For those that leave the terminal, some, maybe most, must present themselves to CBP for processing by returning to the airport.  Others can be processed and cleared remotely, such as over the phone.  This is entirely up to CBP.

In this and another similar incident, the passengers didn't get into trouble, the airline, yes, for sure and was fined.  I don't fault the passengers for leaving.  I would certainly notice the error and stay put, others, even US Citizens, might just assume they don't need to see CBP.

  [1]: http://www.nydailynews.com/new-york/jfk-allowed-passengers-exit-customs-article-1.2507437