Having now done it, it's worth noting several things.
ANA flies between Tokyo and Hachijojima three times daily. The first flight of the day gets you in with about 90 minutes to spare before the ferry, and you have plenty of time to get to the ferry departure.
The ferry to Aogashima is 2.5 hours and is basically a cargo ship with tatami mats. Or you can sit above deck in the open air.
Once you arrive, there's no support, no structure, nothing. The village is on the other side of the island, and you've got a giant cliff in front of you. The ferry departs again almost immediately.
We actually hitched a ride with the local cop, who took us past the campsite (there's only one, and it's on the inner volcano, nowhere near anything). You need to go to the town anyway, to register at the town office.
We rented a car from town and used that to get between town and the campsite.
The ferry claims to run every day except Sunday, but the 'real' schedule on the island shows many days that it's scheduled to skip. On top of that, fog can sit for several days, stopping both the ferry AND helicopter crossings. We were intending on a stay of one night and ended up being stuck for over four nights, finally taking the heli back to Hachijojima. It's a fun but short ride.
The heli is usually around 8.30 or 9 in the morning from the island, depending on previous days. Note it only holds nine passengers, and you have to book. When the ferry isn't running, it's very difficult to secure a booking.
Saying that, it's a great wee island for a few days, and has both camping and ryokan options.