Can a crew member ineligible to enter the US enter US waters and stay onboard during transit?
Whatever it is you're crewing is probably governed by the Pleasure Boat Reporting Requirements, and if not language similar to this would be found in whatever governs it no doubt:
operators of small pleasure vessels, arriving in the United States from a foreign port or place to include any vessel which has visited a hovering vessel or received merchandise outside the territorial sea, are required to report their arrival to CBP immediately ... an application to lawfully enter the United States must be made in person to a CBP officer at a U.S. port-of-entry when the port is open for inspection.
So, no. Entering territorial waters is entering the country far as CBP is concerned.