These rules are stablished in the Chicago Convention on International Civil Aviation(https://en.wikipedia.org/wiki/Chicago_Convention_on_International_Civil_Aviation), that specifies, among others, that:
Article 13: (Entry and Clearance Regulations) A state's laws and regulations regarding the admission and departure of passengers, crew or cargo from aircraft shall be complied with on arrival, upon departure and whilst within the territory of that state.
Also, in the best practices:
5.9 The aircraft operator shall be responsible for the cost of custody and care of an improperly documented person from the moment that person is found inadmissible and returned to the aircraft operator for removal from the State.
5.14 Contracting States shall not fine aircraft operators in the event that arriving and in-transit persons are found to be improperly documented where aircraft operators can demonstrate that they have taken adequate precautions to ensure that these persons had complied with the documentary requirements for entry into the receiving State.
About your particular question, it is the airline's duty to ensure that no people without proper travel documents board the plane, so that check is usually done by airline employees.
Policemen may be in charge of security (X-rays/scanner machines, ensure that nobody boards with another person's boarding card), but they are not airline personal, and are not trained to be so. I guess the policeman thought he was just being helpful by telling you what he believed was right, but in any case it was just his opinion.
In fact, airline employees themselves usually do not know the document requirements (because they depend not only of where the flight departs to/from, but also of your nationality and even more factor) and they mostly check with databases like Timatic.
And, as stated in other answers, no, you cannot sue (or you can try, but you will nothing out of it). There are two guilty parts here: yourself, who failed to travel with the required documents, and the airline, that failed to check those.
The airline may be fined by the Israeli government; you were detained and expelled from Israel and may be requested by the airline to pay for the return flight.