From wikitravel regarding Saigon - Ha Tien:
There are buses to Ha Tien from Ho Chi Minh City's Mien Tay terminal (about 8h).
To get to Mien Tay Bus Station (Western bus station): Take bus No. 139 from Tran Hung Dao Street.
I found this on Tripadvisor, a nightbus to Ha Tien
the bus leaves from the Western Bus Station in Saigon,at 21.00 and gets in to Ha Tien at around 05.30. It costs 185,000 VND.
Additionally I found the company that runs the buses to Ha Tien. That's what's on their website:
Departure: 09h30, 11h00, 15h30, 21h00, 22h00; box office #14 on the western bus station; 175,000 VND a seat
From wikitravel regarding Ha Tien - Kampot:
It takes around 2 hours via motorbike. You can take a minivan - you can buy the ticket at the Tourist Agency at "Oasis Bar" for USD $12 per person. The minivan departs at 12pm every day.