Oh, I did this last month!
A lot of websites and forums have mentioned problems in the past with others failing to get visa on arrival, and being rejected (even Aussies and Kiwis, which for me as a Kiwi is pretty unusual, most places think we're harmless).
However by the time I'd heard this, I was already on the road and had no Iranian embassy nearby to apply to, so I had to hope.
That day, on three separate flights, I know of about 8 people who had to get VoAs (we were all going to a wedding).
- insurance. Travel insurance. The copy of my policy on my phone didn't count, you had to have a paper proof. If not, you are forced to buy insurance, at US$15. Not too bad, just annoying.
- paperwork. On arrival you'll apply at a window, and go pay for your visa on arrival.
- proof of address of where you're staying (initially, not every night). We had a hotel booked, and had their details on hand.
- money. The VoA costs cash - for us it was expensive, around US$140.
- proof of what you're doing in Iran. This is likely the hardest part. For us we had our friend's phone number, our hotel's phone number, and had them on standby to call. One group got thoroughly checked, with every number they had being called, while two of us on my flight just got stamped happily and told to enjoy. Your mileage may vary.
- we didn't have an invitation as such, as some others have claimed, but possibly if we weren't visiting a friend for his wedding, it may have been different.
- to be clear, we had accommodation for the first ... three nights, and nothing booked after that. We ended up travelling from Tehran to Shiraz, Yazd, Esfehan, Rasht and back, and just booked accommodation as we went. And trains/buses. None of that was required at the border.