However in this case, the foreign country hasn't published the rules, so there were no way the travelers or the airline could have possibly known about these changed rules.
This statement that Austria didn't publish the new rules beforehand is simply not true.
It was published in different news sources on the 23rd of December, but it seems, not widely in the British press.
Unfortunately, the Wayback Machines seem to have started their holidays after the 22nd of December since that is the last date that one can retrieve anything from the relevant government sites:
The BBC report (2021-12-28: BBC News - Austria's Innsbruck airport denies 110 Britons entry over new Covid rules) was published over 24 hours after similar reports in the Austrian, German and Italian media.
So this looks more a holiday season reporting problem.
Would these passengers have any standing to recover any costs? Obviously not from the airline, but what about the destination country in some way? What about travel insurance?
Since there is at least one British news report (see below), quoting the new Austrian regulations, 2 days before they came into effect this is unlikely for both cases.
The only British news source that I can find from the 23rd of December is:
Austria travel restrictions and latest advice | The Independent.
Helen Coffey, Lucy Thackray 5 days ago [as of 2021-12-28]
Austria has once again tightened its rules for UK travellers, after adding the country to its “virus variant list” (virusvariantgebiete) of high risk destinations.
The rule change comes in from Christmas Day and will affect many British winter holidays, especially in the ski sector.
...
Can Britons travel to Austria?.
Yes, but only some will avoid lengthy quarantine.
As of 25 December, the UK is being added to Austria’s ‘virus variant list’, a list of high-risk destinations for the omicron variant.
This means that only travellers with a proof of two vaccine doses and a booster jab, along with a negative PCR test result, will be able to avoid quarantine in the country - everyone else must quarantine for 10 days.
The test (PCR, LAMP or TMA) must be taken within the 48 hours before your time of arrival.
The new rule comes in from Christmas Day and will mean many cancelled holidays for families whose younger members have not yet received a booster - some may not even have received two jabs.
The Austrian and German press also reported the changes extensively on Thursday the 23rd:
Österreichs Einreisebestimmungen - news.wko.at.
23.12.2021, 16:35.
Übersicht Einreiseverordnung NEU per 25. Dezember 2021.
...
NEU per 25.12.2021.
Die Staaten Vereinigtes Königreich, Niederlande, Dänemark und Norwegen wurden in die Liste der Virusvariantengebiete (Anlage 1) aufgenommen. Die Einreise aus Virusvariantengebieten und -staaten (Anlage 1) ist grundsätzlich verboten. Dieses Verbot gilt z.B. nicht für EU/EWR-Bürger. Deren Einreise ist nur mit einem negativen PCR-Test zulässig. Es besteht jedoch grundsätzlich eine Quarantäne- und Registrierungspflicht. NEU ist, dass für alle Personen, die einen Nachweis einer „Booster“-Impfung sowie einen negativen PCR-Test (ACHTUNG: in diesem Fall darf der negative PCR-Test nicht älter als 48 Stunden sein) mitführen, die Quarantänepflicht entfällt. Die neue Verordnung gilt vorerst bis 31. Jänner 2022.
Overview of entry regulations NEW as of December 25, 2021.
...
NEW as of December 25th, 2021.
The United Kingdom, the Netherlands, Denmark and Norway have been added to the list of virus variant areas (Appendix 1). Entry from virus variant areas and states (Appendix 1) is generally prohibited. This prohibition does not apply to EU / EEA citizens, for example. Their entry is only permitted with a negative PCR test. However, there is basically a quarantine and registration requirement. NEW is that for all persons who have proof of a “booster” vaccination as well as a negative PCR test (ATTENTION: in this case the negative PCR test must not be older than 48 hours), the quarantine obligation does not apply. The new regulation is initially valid until January 31, 2022.