The rules apply based on the country the incident occurs in, AND the nationality of the airlines.
The US rules apply to all flight FROM the US, AND to all flights operated by a US-based airline (regardless of the departure country)
The EU rules apply to all flights FROM the EU, AND to all flights operated by an EU-based airline (regardless of the departure country)
This means that multiple rules can apply, in which case the higher of the two requirements will apply.
As your flights was from the US, the US rules will apply. If it was operated by an EU-based airline, then the EU rules would also apply. This does not mean that you get the benefits from the two added together, but the higher of both.
Under US rules, the denied boarding compensation would depend on the ticket price paid, but would be up to $1,350. Under EU rules, the denied boarding compensation would be €600 or around US$700.
So if the airline you flew was a non-EU based carrier, then the US rate would apply. If it was an EU-based carrier, whichever was the higher of these two amounts would apply (likely the US rate, but it would depend on the ticket price paid)