Non-insular countries bordering the Caribbean Sea are not included unless they are "territory or possessions" of the UK, the Netherlands, or France.
First, the statute establishing the visa waiver program, 8 USC 1187 has this to say on the matter:
(8) Round-trip ticket
The alien is in possession of a round-trip transportation ticket (unless this requirement is waived by the Secretary of Homeland Security under regulations or the alien is arriving at the port of entry on an aircraft operated under part 135 of title 14, Code of Federal Regulations, or a noncommercial aircraft that is owned or operated by a domestic corporation conducting operations under part 91 of title 14, Code of Federal Regulations).
At some point, someone realized that this is overly restrictive, so they issued a defining regulation, 8 CFR 217, which says at 217.2
Round trip ticket means any return trip transportation ticket in the name of an arriving Visa Waiver Pilot Program applicant on a participating carrier valid for at least 1 year, electronic ticket record, airline employee passes indicating return passage, individual vouchers for return passage, group vouchers for return passage for charter flights, and military travel orders which include military dependents for return to duty stations outside the United States on U.S. military flights. A period of validity of 1 year need not be reflected on the ticket itself, provided that the carrier agrees that it will honor the return portion of the ticket at any time, as provided in Form I-775, Visa Waiver Pilot Program Agreement.
It also says
(c) Restrictions on manner of arrival -
(1) Applicants arriving by air and sea. Applicants must arrive on a carrier that is signatory to a Visa Waiver Pilot Program Agreement and at the time of arrival must have a round trip ticket that will transport the traveler out of the United States to any other foreign port or place as long as the trip does not terminate in contiguous territory or an adjacent island; except that the round trip ticket may transport the traveler to contiguous territory or an adjacent island, if the traveler is a resident of the country of destination.
I think that someone at United Airlines got confused about "contiguous territory or adjacent island," which includes Caribbean islands, and somehow morphed it into a designation that includes "territory adjacent to the Caribbean," which it does not (except for Mexico, of course, which is included not because of the Caribbean but because it is contiguous with the US).
So what are the adjacent islands? They are defined at 8 USC 1101(b)(5):
(5) The term “adjacent islands” includes Saint Pierre, Miquelon, Cuba, the Dominican Republic, Haiti, Bermuda, the Bahamas, Barbados, Jamaica, the Windward and Leeward Islands, Trinidad, Martinique, and other British, French, and Netherlands territory or possessions in or bordering on the Caribbean Sea.
(As far as I am aware, there is no longer any non-insular "territory or possession" of any of those three countries that is adjacent to the Caribbean sea. French Guiana is too far east; it's on the Atlantic Ocean.)
Since Costa Rica is not a possession of any other country, much less of one of those three European countries, you should be all set. Also, if you're a resident of Costa Rica, it wouldn't matter even if it were somehow taken to be included in "contiguous territory or adjacent islands."