Getting from the airport to the city centre on the train is very easy, but it isn't very cheap. The airport rail link is quite new, and there's an extra fee for using the airport stations. As such, an adult single ticket from the airport to the city circle (the main city centre stations) is $15. If you have a MyMulti pass (city wide public transport ticket), there's an additional $11.80 fee for using the airport stations. For three of you, you're looking at $45 to get into the city.
However, if you have time and not much luggage, there's two other options for getting into the city by public transport cheaply. One it to walk from the airport to Wolli Creek station (following this map), then take a regular city train in for $3.20 each. Cheap, but means walking for about 30 minutes with all your bags.
The other is to take the 400 bus (Bondi Junction to Burwood via Airport) from to Burwood, and take the train from there (I think it'd be $2 for the bus, then $4 for the train). Note - don't try to catch the bus to Bondi Junction and change to the train there, as it takes an absolute age (~1 hour!). The bus+train via Burwood is only a little more money, takes about the same time as the walk to Wolli Creak, but doesn't involve walking with bags.
I believe you'd be looking at about the $50 mark for a cab into the city centre.
So, depending on how much of a saving there is on picking up the car from town vs the airport, and how much faff you're willing to put up with on public transport, it may or may not be worth it. My experience of morning traffic from going from near the airport, into the city centre is that it's pretty slow going. Parking is expensive, and there's plenty of one way streets and restricted turns to catch you out if you've not got a good map. With that in mind, it might be safer to make your way into the city centre, get over the jetlag whilst seeing the things in the middle, and only get your car once you're ready to head outside the city!