Since air travel between those destinations is expensive, I was looking for some cheaper alternative.
I would highly appreciate if anyone could advise me on this.
Apparently there are no direct ferries: http://www.tripadvisor.com/ShowTopic-g294217-i1496-k936500-Travel_by_ferry_from_Hong_Kong_to_Taiwan-Hong_Kong.html
It would be unlikely to save any money: see http://www.rome2rio.com/s/Hong-Kong/Taipei.
A mid week round trip ticket is around US $200 on Hongkong airlines or China Air. Maybe US $250 on Eva. Cathay Pacific is significantly more expensive but it's by no way the only game in town. It's hard to see how a ferry could be cheaper than US $200.
Yeah, there are a few options, but air travel is still likely to be the cheapest, as it's a long way to go by ferry. However, for comparison:
shows options including combinations of bus, train, ferry and flights. It still unfortunately looks like a flight is the likely cheapest option.