I would like to go by road to Mecca, either by car or by public transport trains, has anyone tried the route? I have got information on how to travel up to Istanbul in Turkey but from there it is not that straight-forward / easy to reach Mecca, Is there a train service from Istanbul or a possible road journey to reach Mecca from Istanbul.
First, it will not be "straight and easy": the Middle East is full of unstable countries, underdeveloped infrastructure, political and diplomatical issues between neighbor countries and bureaucracy. But it can be interesting to travel these routes and discover less-traveled places and meet locals nevertheless.
From Europe to the Middle East
The first part of your trip can be done quite easily by train in Europe (see also InterRail/EuRail/RailEurope options) and to Syria until Damascus or Iran until Tehran. It's easy to do by car too, and you won't have to pay huge taxes for your car in Europe or Turkey. In the other countries, import taxes on foreign cars may be very high and make it more interesting to rent a car (and possibly a driver) locally in each country or take public transportation / taxis / buses.
You can travel by bus easily, and by train to Bandar Abbas where there are ferries to Sharjah and Dubai (but the service seems to be quite irregular). You can also take a plane to cross the Ormuz straits. Another ferry route is Bushehr-Dammam, but Bushehr is not reachable by train (buses and flights exist though). A map of train routes can be found on the website of the Iranian railways.
Beware of the security / diplomatic situation between your home country and Iran.
This is of course not recommended at the moment.
From Damascus (where you can arrive by train, car or bus from Turkey), go to Amman by train (notoriously slow, taking up to ten or twelve hours for a trip that a service taxi does in less than three hours on the road, so it is really interesting only for those looking for a "train experience"), car or bus. There are no other passenger railway services in Jordan, so you'll have to take cars/taxis/buses from Amman to Makkah.
This is suicidal at the moment, and most borders remain closed or with big restrictions. Don't attempt it.
Avoiding Israel, Iran, Iraq and Syria: through Egypt
While this will require from you to take boats or flights, there is also the possibility to make a detour through Egypt and to cross the Red Sea with a ferry, avoiding Israel. You would have to take a ferry from Greece to Cyprus and from Cyprus to Egypt. As of now, the line between Limassol (Cyprus) and Port-Said (Egypt) has been discontinued, even for pedestrians. But it might reopen at some point for pedestrians. The Louis Cruise line is also suspended. So right now your only option would be to be on a cargo ship: it won't be easy.
The trickiest part of your trip, apart from instable countries like Syria or Iraq right now, is probably Saudi Arabia itself. The only way you can cross the country by land is on a (very rare) tourist visa, or a work visa and permission from your employer to wander around the country. If you come on a Hajj or Umrah visa, you will most of the time have to arrive by air in Jeddah or Madinah, with your pilgrims' group. Some pilgrims arrive by land or by sea, but they are coming from nearby / poor countries where air travel is not an option for everyone, and they're still part of a group. Onward travel to Riyadh or other Saudi Arabian cities is usually not permitted during the Hajj. During Umrah, visitors may obtain permission after arrangements have been made with a travel agency to travel to other cities in Saudi Arabia. Umrah visa holders may only visit Saudi Arabia outside of the annual Hajj period.
Inside Saudi Arabia, almost all your land travel will be by car, taxi or bus. The only rail connection today (apart from the Makkah metro) is Dammam-Abqiq-Hufof-Riyadh, allowing you if you have the necessary visas to connect with the ferry from Iran. There are railway expansion projects though, even if they're not as large as the Ottoman-era Anatolian (Istanbul-Ankara), Baghdad (Konya-Baghdad) and Hejaz (Damascus-Mecca) railways yet (the only other neighbor country involved in today's projects being Jordan).
Israeli visas or stamps in your passport, or an Israeli birthplace will have your visa request or entry denied. You may be able to have you entry stamp to Israel put on a separate sheet of paper, but it's never ever guaranteed to work so prepare for changing plans if you go that route!
Note that tourist visas to Saudi Arabia are not typically granted, and I didn't hear of tourist visas for individual travelers yet. This may be changing in the future, in light of the National Tourism Development Project and the recent acceptance of large tourist groups. You can get all information regarding visas and associated fees from the Saudi Embassy website.
Note also that unless traveling with a husband or male relative, women will have the most difficult time getting a visa into Saudi Arabia (it's easier if they are old, but not guaranteed nevertheless). Women will be required to wear an abaya (full face and body cover) once they get off of the plane and in all public places throughout their stay. Men should not wear shorts in public or go without a shirt (think of taking clothing like the one you require for visiting a mosque for your whole trip inside KSA). Men and women who are not relatives are not allowed to travel together in Saudi Arabia if the women's guardians are not with them at all times.
Finally, you have to be a Muslim (and prove it through a certificate delivered by your imam if your name doesn't sound Muslim) to enter Makkah and Madinah.
Iran is the safest way in Middle East when you want to avoid Iraq, Syria and Israel.
As I said in my answer to Quickest way from London to Dubai without flying?, there is a direct train from Istanbul to Tehran. From there, you can go to Bandar Busher by bus, then take a ferry to Dammam. Finally, you can take a bus to go to Mecca.
Even assuming you are Muslim and can legally visit the city of Mecca, an overland trip between Istanbul and Saudi Arabia is not practical at this time due to the civil war in Syria and instability in Iraq.
Should the situation in Syria improve, then it is fairly straightforward. Drive from Istanbul to Antakya/Kilis/Şanlıurfa/Nusaybin and cross to Aleppo. From there, drive through Damascus and then to Amman. Once you are in Jordan, there is a border crossing into Saudi Arabia near the Red Sea port of Aqaba. However, I doubt this will be possible for the foreseeable future due to the fighting in Syria.
Alternatively, you could cross into Iraq from Turkey and travel south by taxi into 'dangerous' Iraq and then onwards into Jordan. Taxi drivers do these kind of trips all the time but even if you arranged for security (or are able to blend in), you'd still need to deal with the fact that the visa issued at the Turkey-Iraq border is not valid in the parts of Iraq not controlled by Kurdish authorities.
The closest to an overland journey you could manage these days would be to use ferries between Greece or Turkey and Cyprus, from where there are ferries to Israel and from there transport to Jordan. Needless to say, there are implications to using this route inasmuch as Saudi Arabia won't let you in with an Israeli stamp in your passport.