From the State's Department Visa Waiver Program webpage:
The Visa Waiver Program (VWP) allows citizens of participating countries* to travel to the United States without a visa for stays of 90 days or less, when they meet all requirements explained below. Travelers must be eligible to use the VWP and have a valid Electronic System for Travel Authorization (ESTA) approval prior to travel.
(Note that it says “and”, still having a valid ESTA does not exempt you from fulfilling the other requirements for eligibility at the time of entry.)
According to the same page, one requirement for eligibility is:
Previous Compliance and No Prior Visa Ineligibilities
If you have had a U.S. visa before or previously traveled to the United States under the VWP or another status, you must have complied with the conditions of previous admissions to the United States, and you must not have previously been found ineligible for a U.S. visa.
There are some subtleties regarding what counts as “being found ineligible” and you haven't provided any details on the reason cited to refuse your O-1 visa application but it seems you might not be eligible for the Visa Waiver Program anymore.
If you have in fact been found ineligible, this means that not only is there “a chance” to be denied entry but in fact that you should be denied entry if someone finds out about your earlier visa denial. You probably also would not be able to get a new ESTA without lying on the application form (which would create a new sets of liabilities for you, up to a potential ban from the US).
Also note that the rule, like other rules of this kind, apparently refers to you as a person, not to your passport. Using another passport/citizenship to try to get around checks does not make you eligible for the Visa Waiver Program.
Your latest comment suggests it might in fact still be possible to travel on an ESTA in your case. As explained on the Previously denied a visa or immigration benefit page from the CBP website, the best course of action would be to ask a US consulate about that before attempting to enter the US.