Yes, quiet rural spots in South Africa are absolutely the places you should most be careful, and more so than other countries. Several guidebooks recommend avoiding such places completely although it depends somewhat on area. I think that advice is a little out of date, crime in South Africa is much lower than it was, but those safety warnings definitely do apply outside the cities. One book even advised against stopping at roadside picnic tables. I'll add some quotes when I have these books to hand, I'm having trouble finding the right sections online.
It's not just a racial thing as one comment suggests, although if you stand out that obviously doesn't help. Plenty of Basotho people I met in Lesotho complained about how on-their-guard they felt they needed to be in South Africa compared to Lesotho. Some places people will be extremely welcoming to outsiders, other places outsiders are seen by many as fair game (and some places have a mix of both), the problem is you can't easily tell which is which.
Some tips for how to do this kind of independent exploration more safely:
- Each area you visit, get up to date local advice. People like guesthouse owners will usually be able and happy to advise on where is safe to visit and where is best to avoid, even literally marking safe and dangerous districts and areas on your maps.
- Lesotho and Swaziland have much lower crime rates and, importantly, different patterns of crime (more towards petty crime, very low rates of things like ambushes or carjackings), and are generally considered reasonably safe for this sort of thing in rural areas, particularly Lesotho. If you're exploring the North East of South Africa and enjoy this kind of spontaneous off the beaten track travel, I'd strongly recommend taking a route through either, especially Lesotho (with the caveat that you may need to check your vehicle is capable of any particular road before setting out - you can pick up a free "Visit Lesotho" map in many places which have all the country's roads graded including "4x4 only" clearly marked)
- South Africa's national parks and wildlife reserves are huge, well developed, and generally very safe. Most you can drive and explore at your leisure, and while many are very touristy some are reasonably off the beaten track. Some you can't leave the car due to dangerous wildlife except in designated spots like lodges and lookouts, but it's still great to just explore.
- Try to avoid driving at night if you can (though this one is more for the much higher risk of accidents than the also real and higher than usual risk of crime)
- Three tips lifted straight from Lonely Planet's website. There's much more in their physical book but I don't have my copy to hand:
- Leave your car in secure parking at night and avoid parking in secluded areas during the day.
- Don't leave anything valuable in your car, or give the impression that you are on a road trip with bags in the boot.
- One of the greatest dangers during muggings or carjackings (most common in Jo'burg) is that your assailants will assume that you are armed and will kill them if you get a chance. Stay calm, and don't resist or give them any reason to think you will fight back.
While that last point is especially true for quiet city streets, it's also the sort of thing that could happen if you pull up in the wrong town, village or township. Get local advice and take care, but don't let it stop you enjoying the many great places in South Africa that are safe.