The answer will depend on the destination country, and to some extent the airline.
Some countries (for example, the US and Australia) have electronic systems that the airlines must use to confirm a passenger meets the entry requirements for the country - such as holding a valid visa or equivalent documentation (eg, ESTA). At check-in, the airlines computers will query that system and confirm the passenger is good to travel. If they are unable to verify the passenger has the required documentation then they will generally fail to check the person in, and require manual intervention by a check-in agent to fix the problem. (eg, maybe the passenger has 2 passports and it's the second one that allows visa-free entry).
Other countries rely on physical stamps in the passport which need to be checked by the airline staff. Depending on the airline they may choose to confirm you have the required documentation before they issue a boarding pass (which often means you will only be able to check-in for those countries at the airport if you require a visa to enter), OR they may choose to flag your booking in a way to let the staff at the gate know that they need to check this before boarding.
The exact process for this varies between airlines. You may sometimes hear passengers at the gate being called to the podium for a "documentation check", which is the staff carrying out this check. Some carriers will put text or a stamp on the boarding pass to show that this process has been carried out. For example, United Airlines currently puts "Travel Ready" on the boarding pass to show that all relevant checks (including visas/etc, COVID-19 requirements, etc) have been successfully carried out. Other carriers may put "DOCS OK" to confirm that the documentation check has been carried out.
The process of determining where a visa is required also varies between airlines, but most major airlines use a system called Timatic, which is normally automatically queried during check-in to determine the exact documentation requirements for a trip.
Specifically for Visa-On-Arrival, from the airlines perspective if a passenger is eligible for Visa-On-Arrival (which is something Timatic will show if they are) then the passenger can be treated as not requiring a visa for that specific country, and there is nothing for the airline to specifically check, beyond the fact that the passenger meets the requirements for obtaining a visa on arrival.
