Absolutely! And not only you can, it is one of the best ways to explore the City! My favourite place to start the journey is from Hyde Park - right by the Serpentine Lake is a nice place to eat a sandwich and start skating, while you see some people dancing on rollers and quads.
There are also some places to rent inlines in case you can't take yours with you near Hyde Park (Slick Willies and Club Blueroom come to my mind). They usually provide skate pads as well.
You may want to attend the LFNS (London Friday Night Skate) weekly meetings -on Fridays and Sundays-, which are very well organised and safe, as they always have marshals holding the traffic while the rest of the people just enjoy the ride. It is run by volunteers, and the people is just great. I can't recommend these rides enough.
If you want to go solo, you can either go by road or on the sidewalks (but please be careful if you do so). The police officers will not disturb you unless they find that you are annoying the rest of the pedestrians.
By the way, keep looking the road ahead of you, as London roads are far from perfect and there may be holes in the ground or even rubber cables crossing the road side to side.
Some other must-skate places in London are Greenwich park, Kensington and Chelsea boroughs and Oxford St./Regent St. (these one on Sunday mornings, as they are a bit too crowded otherwise).