I understand that ESTA travellers are allowed to travel and can also go the Automated Passport Control (APC) in certain airports. But once I have arrived in a state (e.g. NY) and then would like to go to another state e.g. IL - How does this work? Do I need to treat this as an international travel i.e. do the ESTA based checks again?
No. It's a domestic flight. There's no immigration control. Only passengers arriving from international destinations are sent to the immigration hall. When you check in for the flight, you'll need your passport, but only for the purpose of identification (actually, there are several other documents that can be used for identification, but most ESTA travelers won't have one). Your immigration status is very unlikely to be an issue unless you're flying from an airport near the Mexican border. Even in the unlikely case that it becomes an issue, ESTA will have nothing to do with it; the relevant record is your I-94 record in the CBP database.
If you try to cross the US-Canada border, you'll know what the States didn't want to have between them. The idea of no border controls is an important part of American identity. There is simply no immigration control at State borders. None.
On the train, the border whisks by and you never know it's there.
In air travel, you have TSA security to deter hijackings, but that's nothing like customs and immigration. The only question I've ever been asked is why I didn't know to take my laptop out of the bag, and they did look for manner and presentation to see whether I was being sneaky or merely uninformed.
By highway, the roads are wide open. Contrast borders with Schengen-area nations and British Countries:
(Ohio's sign was probably knocked down). On the larger roads (e.g. Interstate freeways) they will have much nicer signs. They will also have a "Welcome Center" at the first or second exit, where you can stop to get free official State maps, and brochures of area attractions. The Welcome Centers are not mandatory, unless you are obsessive about collecting free maps.
Texas's border controls go a step further. They add this sign:
A few states, such as California, will have inspection stations near borders for agricultural checks - all they ask about is if you have certain foods they are concerned with, such as fruits and vegetables (carry invasive insects) or northern pike.
Checkpoints not related to State borders
Relevant to immigrants is US border patrol checkpoints. Some are interior checkpoints, and being Federal, they completely disregard state boundaries. Some are permanent, others are improvised.
Police will set up checkpoints for a variety of reasons, usually drunk drivers. This has been challenged unsuccessfully in court.
Domestic TSA aircraft security (and the more subtle security on rail) is not about state borders.
Any police encounter is a risk for overstayers and people with arrest warrants, but it really depends what they're looking for.
Some highways and most ferries charge tolls, and have toll-booths, but most prefer you'd bypass the human and use an automated toll payment system. The only question you are likely to get is "Do you have any bills smaller than a $100?"
California and a few other states will seasonally have tire-chain inspection stations - because when someone says "California", the first thing you think is tire chains!
Sources: Wikipedia and mediawiki