Jozza'a quote above is on so many of the 'crazy' travellers' Facebook profiles that I encounter. I'm in southern Russia at present after a 34 hour train journey, and I'm trying to work out how to get into Kazakhstan. I don't speak the language, don't know where I'm going, and am on my own. And you know what? I haven't had any problems, no crime, the train runs fine (exactly to time), and in general people are friendly and helpful, even if you can't understand each other.
A friend cycled from London to New Zealand over 20 months. She was on her own cycling through places like Iran, Afghanistan, India - all 'hard' countries, some very scary. Again, she was fine. She had some scares, sure, but in general she made it through with amazing stories.
My personal quote is that as long as you survive a bad experience, it makes a great story and adds to your character.
Another friend (who I'm currently chasing) is in Uzbekistan. He's riding a 85cc scooter from London to Mongolia. He's crashed, broken down, lost parts - and people have welded his scooter together for nothing, hunted out spare parts for him, and always refused to take money - they're happy to help out of the goodness of their hearts.
It's something you start to realise - every country has some crime, some bad people. But if you're careful, in GENERAL, people everywhere are basically good, and happy to help a fellow human being.
Don't be scared, be cautious. It's a fantastic world out there - it's scary, but it's exciting. Go explore it!