If your ESTA application is denied, you cannot enter the United States under the Visa Waiver Program (VWP).
However, this does not mean entry is completely impossible.
In such cases, you must apply for a visa (B1/B2, etc.) directly through the nearest U.S. embassy or consulate.