When I travel I usually just withdraw money from an ATM. Does that mean that in this case, I should be bringing lots of USD to exchange?
Yes, it means exactly that.
Does it have to be USD? Do they accept Australian dollars at all?
You might be able to find someone who does, but it will be much harder and the exchange rate will not be as favorable. USD is best, though Euros and Brazilian Real are also commonly accepted. Large notes are better than small ones.
I also heard it is technically illegal to do this, do I need to take any precautions?
Not really, especially not as a tourist. Obviously, you should not follow some guy who promises super special exchange rates into a side street in a shady part of Buenos Aires, but there's no reason to - in the tourist hot spots of BA, traders will find you, and they operate quite openly.
Update: In December 2015, the new Argentinian government removed the exchange restrictions, so the official exchange rate is now based on real market forces and the black market has lost its advantage.