Another reason, beyond the excellent ones already discussed here, is that public transit is typically subsidized by government, often both by direct annual contributions to the transit agency's budget, and by issuing debt for capital expenses that is slowly paid off by some combination of the agency and the government.
For operating expenses (not counting the cost of actually building infrastructure or rolling stock), the extent of the subsidy can be roughly measured with the Farebox Recovery Ratio, which is the percent of operating expenses paid for by fare revenue. As you can see, almost all systems outside the ultra-high-density operators in Asia have rather low farebox recovery rates, often under 50%.
What this means is that any new expansion in service may well cost more to operate than it makes in revenue. This is normally accepted, because that expansion comes with the public good of allowing more residents (taxpayers) to access the system, reduces traffic congestion and pollution, and improves property values (and thus raises property tax revenues) because homes with good transit connections are often more valuable.
But for an airport link, taxpayers tend to balk at subsidizing comparatively well-off travelers headed to/from the airport. There is often a desire for such links to pay their own way in operations expenses, and ideally recoup their construction costs, and so when agencies go to set fares, they often price the airport segment considerably higher than other trips of equivalent distance.
As an example, BART's Oakland Airport Connector in Northern California opened in late 2014. Transit advocates encouraged the agency to set a high fare, so as not to divert funds that are used to serve commuters to the airport. The agency eventually set a fare of $6 (just for the 3.2 mile Connector; the rest of your ride still costs the normal amount). Even at this high fare, there's considerable question about the Connector's ability to pay for itself, let alone pay back the hundreds of millions of dollars it cost to build.
TL;DR: Transit is usually a lot more expensive than you think, but is often heavily subsidized. When the subsidies are reduced or eliminated, as often happens with airport links, the fares look far more expensive by comparison.