I am planning to visit San Francisco but I am unsure about public transport reliability and safety, being a lone female traveller.
How good and safe is public transport in San Francisco?
It is always hard to tell about safety because people have very different criteria about safety.
Having used the public transportation, buses in particular, in San Francisco, including poorer neighborhoods, I would consider it safe. I am a tall white man though, so the feeling of security might be different.
San Francisco buses are the places that show the variety of the population the most. You can see rich Silicon Valley people with fancy clothes as well as homeless people smelling pee and carrying their home on their back. All this wide range of the society circulates peacefully on the buses, with few altercations in my opinion. The bus drivers in particular are very patient, and seem to have authority on the buses they drive.
At night, you might want to avoid the buses a bit more, depending on your feelings. The bus 91, which tours around the city, is a moving hotel. The bus is filled with homeless people sleeping on the seats. This is still safe to me, no fight or whatever ever happened while I was on the bus but you might not feel as comfortable.
I do not know much about the BART, but I heard incidents may happen at night. Well like everywhere, there are always risks. Regarding the Caltrain, as @Spehro mentioned, it is safe, filled with commuters in general - some with expensive bikes. But I doubt you will use it to visit the area.
Overall, you will be safe, even though I would recommend you not to show off your valuables too much (it is a rule that applies to every public place in the world, in my opinion). If you are not comfortable with poverty, you can as well rely on cabs, faster - but more expensive. My opinion is that I love San Francisco buses for seeing all the society there, I have never seen that anywhere else. I consider it a tourist attraction by itself.
In 2011, US News and World Report rated San Francisco 13th best in the US for public transit. I agree with Vince that it's pretty safe overall but you might want to avoid certain lines, particularly at night. (But like Vince I'm also a large white guy, so...)
Here's a list from 2010 of the top 10 bus lines for reported crime incidents. As a city resident from 1996-1998 and 2010-2011, the list fits pretty well with my intuition. According to that list, together these lines account for more than 50% of Muni bus crime, so if you avoid them your odds should be pretty good. As Vince says, most of them won't go where you want to go as a tourist anyway.
I'm a little suprised to see the 38 Geary on there, but that may partly be because it's just so busy. If you want to avoid the 38 but you need to cross town east-west, the 1 California bus and the N Judah streetcar are both decent alternatives. And even the buses on the list are pretty safe during morning and evening rush hour (apart possibly from a risk of pickpockets). I've commuted via the 38 Geary, 44 O'Shaughnessy, 71 Haight-Noriega and 22 Fillmore and seen plenty of solo women riders.
Here's an SFGate article from 2014 on the BART stations that see the most and least crime. You're unlikely to be anywhere near Bayfair but it might be worth avoiding Powell in favor of Civic Center (to the southwest) or Montgomery (to the northeast). Again, though, you're not likely to have any need to be in the Powell Station neighborhood, and the train itself is pretty safe.
Updated to add: I forgot that Powell Street is the closest BART / light rail station to Union Square, and it's also the terminus of the Powell/Hyde cable car, so it's not out of the question a visitor might have reason to be there. But Montgomery Street isn't much farther, and Powell is still safe enough at busy times.
I will offer a slightly different opinion. I am a 6'3" and 220lbs+ white male and San Francisco is on the bottom of my favorite places ( I travel an awful lot ) somewhere in line with Amsterdam. In particular, I never felt unsafe on the (trolley) buses I take but the surface area of the 16th and the Powell BART stations make me very uncomfortable especially in the dark. If I would be single female I would avoid those in the evening. Otherwise... it's just a very busy, "too much" city that I wish I could avoid but work can't let me. I guess it's not unsafe but it's unpleasant.
I was there for Google I/O last year and the public transit seemed very safe to me. I used it to visit the touristy places like Fisherman's Wharf and the Golden Gate bridge from the Civic Center/Market St. area. Like the others so far, though, I too am a tall male.
I would consider the public transit safer than walking, especially at night in some areas, such as the civic center. There are a lot of homeless people around the civic center area and many of them seemed mentally disturbed to the point of belligerence, though others were actually quite friendly. Walking is fine during the day, but I would be concerned about a lone female walking around at night in some parts of the city. I didn't get that feeling at all from the buses, though.
Another option to look into is Uber/Lyft style services. I used Uber once while I was there and it worked very well. They both offer you a free first ride, IIRC. There obviously seemed very safe, though more expensive than the public transit past the free first ride.
For the public transit, you can get a pass pretty cheaply that lets you ride the buses, street cars, cable cars, and subways unlimited for a few days (Three, I think it was?) That's what I used and it worked out very well. This was probably the cheapest thing I purchased in San Francisco. - haha
Every tourist takes a cable car. That's why the fare is triple the standard. (If you are into museums, there is a combined museum transit "City Pass" that may be worth it.) Personally, I like the old streetcars that run up Market Street on the surface to Fisherman's Wharf. Unless you come from the handful of cities with their own historic streetcars (=trams, to much of the world), they're a real nostalgia kick. (There are also modern streetcars that run under Market Street and above-ground away from downtown.)
There are bus routes I would not recommend to tourists, especially ones who might look vulnerable, but I can't imagine any reason a tourist would be taking them.
Let me also mention that the tourist core of SF is compact enough to walk everywhere. If you want to visit the East Bay (e.g., Berkeley), BART is safe, even at night. Most (not all) neighborhoods here "roll up the sidewalks" early, after 9 or 10 p.m., everything is closed.
As an extra note to what the others say, I would like to add that the stops, for example the bus stops has usually homeless people, which seek a place to sit and such.
While this may sound minor, it is not, since homeless people in San Francisco are the most ruthless of their kind (from what I have seen), and especially, some of them turn crazy, screaming for no specific reason, something that frightens me the most!
Notice that I am referring to the stops only, not inside the public transportation, which I have no experience here in SF, fortunately.