I going to take the Oath Ceremony next week, and I am planning to travel to Hong Kong right after the Oath Ceremony. Suppose I am a naturalized citizen of the US, but also a citizen of Hong Kong:
First, can I travel to my home country without US passport but with Hong Kong passport?
Second, people said I cannot go back the US without US passport, thus can I apply the US passport in Hong Kong?
people said I cannot go back the US without US passport,
This is not correct. On one hand you must have a passport but there's no penalty for breaking this rule and more importantly, US citizens can't be denied entry into the USA if adequate proof is provided they are indeed citizens. There is no hard and fast rule here but the USCIS itself says
You will receive your Certificate of Naturalization after taking the Oath of Allegiance. You may use your Certificate of Naturalization as official proof that you are a U.S. citizen.
So then the border agents should accept this. I do not think it'd go as smooth as entering with a passport but it should work. Note that the airline might give you a hard time boarding. See https://travel.stackexchange.com/a/24662/4188 this answer.
thus can I apply the US passport in Hong Kong?
Yes. http://hongkong.usconsulate.gov/acs_passports_adult_first_time_applicants.html The U.S. Naturalization Certificate is mentioned again.
US law says you must leave and enter the US with your US passport, but the US border officers do not actually look at the documents of people who are leaving, so they do not actually enforce that part of the rule.
As for returning, as others have noted, you can get into the country with your naturalization certificate, though you may have to do more explaining than you would have to if you had a passport. I think you could in theory be liable for a fee to gain exemption from the requirement, but I have read on this site that they do not actually impose one.
The biggest problem with returning to the US on the strength of your naturalization certificate would most likely be the airline. They might not accept your naturalization certificate as proof of admissibility in the US, since it doesn't comply with travel document standards, and if you lack a US visa in your Hong Kong passport, they might refuse to let you board.
Therefore, getting your US passport in Hong Kong seems like the safest way to go.
