The word "leave" is here used in the slightly archaic sense of "permission".
Leave to enter is what a third-state visa-free national gets from an immigration official when he is allowed to pass through the immigration control line into the UK. It usually takes the form of an ink stamp in the passport, showing the date of entry plus conditions for the stay (such as no work and how long the stay can be):
(Citizens of EU/EEA states are not formally given leave to enter, but are instead "admitted" under the EEA regulations; this admission is equivalent to a leave-to-enter for many purposes).
Where the UK differs from many other countries is the phrasing it uses for visas. The formal term for a visa that appears in many (but not all) legal contexts is entry clearance; visa applications are processed by "entry clearance officers" and so forth. The UK government tends to use the word "visa" in communication that targets the traveling public, though.
Entry clearances are issued in the form of a passport sticker.
Entry clearance is not exactly the same as a visa, in that it can be applied for (and issued to) non-visa nationals even for short visits where the traveler could just have sought leave to enter at the border. In that case the entry clearance tells the immigration officer at the border that the traveler's documentation has already been found to check out by an ECO, which allows for a much abbreviated landing interview (basically just verifying the traveler's identity) and less uncertainty for a traveler who might otherwise be rejected.
When a traveler has entry clearance, the entry clearance itself usually serves as (or instead of) leave to enter. Instead of stamping a leave-to-enter in the passport, the immigration officer will stamp the entry clearance sticker with the actual date of entry.
Thus, in contrast to, say, US or Schengen visas, a UK entry clearance constitutes an actual permission to enter that border guards are only supposed to override if they positively smell a fish.
The terms 'entry clearance', 'entry certificate', 'leave to enter' and 'leave to remain' are defined in the Immigration Act 1971.
There is lots of information at gov.uk: Entry clearance basics, Immigration rules part 1.