First, US law requires US citizens to enter and leave the country on a US passport. Your plan is therefore illegal, but as a practical matter it might be possible to do it.
You do not identify your other country of citizenship. If nationals of that country need visas to enter the US, you will need to get a visa without the US consular officer detecting the fact that you are American. For example, if you were born in the US, your foreign passport will show that, and they will probably ask why you are not using a US passport. They may also check visa applicants against some database in which you might be listed as a US citizen (US passports issued in the past?). Then again, they might not.
If you arrive in the US showing a non-US passport, the border guard might not notice that you are a US citizen. In that case, you will be treated as a national of your other country, and you might well be admitted.
On the other hand, if the guard does suspect that you are a US citizen, because of your place of birth or otherwise, you will probably be asked why you're not using a US passport. If they establish that you are in fact a US citizen, you will possibly get in trouble for trying to pretend not to be one. At the very least, you will probably have to pay a fee to enter the country without a passport. I don't know what the fee is, but I suspect it is higher than the fee for expedited passport renewal.
IF "back to the country I hold dual citizenship to" refers not to the US but to the other country, then you are much more likely to succeed, as long as you don't try to re-enter the US. But why not fly directly from the US to that country? There's no passport control for people leaving the US, neither in the airport nor at the land border crossings. I have dual nationality and I never show my US passport to anyone when I leave the country -- I check in with my non-US passport.