St Paul's is a great start, the views are amazing. Primrose Hill while not a tall building does offer a free panoramic view of London from the north. If i recall correctly you can see the Gherkin, London eye and BT tower.
Tower 42 has a champagne/wine bar on the 42nd floor and a restaurant on the 24th floor. You usually need to book for the bar, but sometimes they'll let you in without booking if there's space.
The New Zealand society often hosts events(for a small fee) in the penthouse of New Zealand House near Piccadilly Circus it is the tallest building in the area and has amazing views of Westminster the eye, horse guards parade and the backyard of number 10.
Once a year (normally in September) London has an open house where you can get access to many of London's iconic buildings including the Gherkin(at least in previous years). Key attractions such as the Gherkin are ticketed (due to demand, all sites are free so if someone is selling the ticket its probably fake.) so you will need to book in advance. There are many other awesome places that few people know about also on offer, such as old war offices in underground bunkers and unused tube stops. The list changes yearly so this is by no means a sure bet.
If you're resident in the UK you can take a tour up Big Ben by contacting your local MP.