I recently discovered that I will be over-staying my 90 day visa waiver by 2 days. My question is about leaving the country as I fly from NYC to LA before flying home to Australia and I'm wondering that if because the Visa has technically expired will I be allowed to fly from one US state to another or will I be forced to fly directly out of the US from NYC?
There is no requirement for visa to be valid for travelling within the US as long as your identification is valid you will be allowed to board the flight. The problem will come if you decide after flying back to Australia to re-enter the US. More likely then not you will not be able to enter under a VWP, since you have overstayed your visa. Whether or not it results in all out entry ban remains to be seen.
Overstaying the Visa Waiver Program means that you will face more difficulty trying to return to the US. You will be ineligible for the Visa Waiver Program in the future, because you overstayed. You will have to apply for a B-2 visitor visa to visit the US.
If you overstay the time that you were given on the visa waiver program you will start accruing unlawful presence. You may be barred from returning to the US for 3 or 10 years depending on your length of overstay.
So long story short, you'll be breaking US law by overstaying. And that's NEVER a good position to be in. Sure, you'll likely to be able to fly to LA, as they're just looking for ID (eg passport) - you don't go through immigration. But when you exit the country (LA), they're going to be asking questions, and you may find yourself with a flag on your passport, or worse.
Remember, even if you're not ever intending to return to the US, having this on your record may impact your ability to travel / immigrate to other countries in the future as well. Even if you think you won't need it, never say never, and if at all possible, fly earlier, or call the CBP or your consulate and ask for the best possible course of action.
Overstaying your visa duration is definitely a frowned upon offence and can damage your chances of being allowed reentry into the United States. Whilst doing the visa wizard on the immigration visa website one of the questions is: "Have you ever overstayed a visa, even by one day?"
The United States takes such violations VERY seriously, so I would tread carefully. I think your situation can't really be answered on this site, I recommend speaking with an immigration lawyer who can assist with obtaining the right information and what your options are.