One obvious and common interpretation is that people going to Moscow (from London or Tokyo) are often prepared to pay for the additional convenience of a non-stop flight (especially if they are flying on an expense account). On the other hand, people flying between Tokyo and London have no reason to prefer flights with a layover and require another incentive in the form of a cheaper fare.
Similarly, the airline wants to charge as much as possible to each individual client. That means charging higher fares to business travelers and for non-stop flights where they don't face competition. Reducing fares on the very same flights if and only if they are combined with a layover is a way to fill them up and get some money from price-conscious travelers without damaging the revenue from business travelers (the technical term for this is “price discrimination”).
But as @choster noted, airline yield management is often counterintuitive and fare rules can often produce side effects so you shouldn't expect any “rule” to hold in all situations.
In practice, I am not aware of any surefire way to identify potentially free or cheap stopovers. You can try to read fare rules but it's not especially quick or practical. And, once you have identified a potential layover (perhaps through a regular “round-trip” search), you can enter the details of the connection you want using the “multi-city” feature (i.e. not as separate tickets) to see how much it would really cost.