I'm allowed to stay in Taiwan visa free for up to 14 days. Is it okay if my ticket will be one with arrival say on May 1 and departure on May 30, but I will go to Hong Kong for a week in between (through a separate ticket) and will not stay in Taiwan for more than 14 days at once?
There was no problem with this. Of course I was asked to show my ticket to Hong Kong when I was checking in in the US, as Michael Hampton points out, and also to show my ticket to the US as well as hotel reservations in Taiwan before boarding to my flight to Taiwan in Hong Kong.
You will almost certainly need to show your onward ticket to Hong Kong to the airline at check-in/gate when you first board the plane to Taiwan, so that they know you will not be there more than 14 days and thus will not deny boarding.