4 years ago i entered the US under the VWP and ended up overstaying by an extra 30 days. On my second visit I was almost denied entry but they told me the next time I want to visit I'll need to apply for a B2 visa. My question is: the next time I apply for a B2 visa and get it, can I change it to a student visa once I arrive?
Yes, you can. Here's Rutgers, The State University of New Jersey instructions on doing so. I am sure your F-1 "provider" will have similar.
Edit: Here's USCIS of the same. It lists the categories that are not eligible for a change and B1/B2 is not among them and says you need to file a Form I-539.