I am currently studying on a J1 visa which expires in May. I am aware of the grace period of 30 days, but I want to stay in the USA for 90 days after my J1 expires. Can I apply for an ESTA while in the USA and stay in the USA, or do I need to leave the USA and re-enter on the ESTA?
ESTA is merely travel authorization.
What you wish to do is extend your stay by 90 days.
If you wish to do this using the Visa Waiver Program, you will need to leave the country, and re-enter under the VWP conditions (this will usually require an ESTA).
You can apply for an ESTA within the US, however, you still need to re-enter the county under the Visa Waiver Program.