For travel TO the US you can find the detailed rules here. https://travel.state.gov/content/travel/en/international-travel/before-you-go/covid-19_testing_required_US_Entry.html
The sample must be taken 3 calendar days (NOT 72 hours) before departure. Departure is in most (but not all) cases the first leg of your flight. Example: if you are flying from Munich to Frankfurt to New York (on a single ticket with normal layovers) the "departure day" is determined by the time your flight from Munich is scheduled to depart.
So if you first leg departs Friday, the earliest you can a test is any time on Tuesday. It doesn't matter what time on Tuesday you take the test or what time on Friday your flight departs. Flights departing at 23:55 are great, flights departing at 00:10 not so much.
Things get a little more complicated if you take your test in a different time zone than your departure airport.
For travel FROM the US, the rules of the destination country and potentially the transit countries apply. These vary all over the place, so there is no single answer.