This Havataş bus appears to be new (it's certainly new to me), so I can't comment on their service.
Istanbul's public transit does not run regularly at night but the buses to Atatürk Airport are fortunately one of the exceptions. If you use the clunky (in terms of interface and translation) İETT How to go there? website, you'll find that there is a public transit option involving a change of bus that will get you to Yenikapı on time:
ATATÜRK HAVALİMANI 03:30:00 96T ÖMER.DÜRÜK.CAMİİ 03:50:00
ÖMER.DÜRÜK.CAMİİ 05:36:00 96T YENİKAPI İSTASYON 05:57:00
You can only search on exact strings with correct accents so "Ataturk" and "Yenikapi" won't give you any results.
The trip as quoted by the website will cost you four bus tickets which you'll have to find, unless the bus drivers accepts cash (technically they don't sell tickets on the bus, but most drivers will actually do so by carrying electronic tickets which they sell privately).
You might want to look into whether the 03:30 96T from
ATATÜRK HAVALİMANI doesn't just continue through directly to
YENİKAPI İSTASYON as I expect it does, or if the 96T from
ÖMER DÜRÜK CAMİİ at 05:36 didn't start at the airport. If nothing else, you'll save yourself 2 bus tickets and an hour and a half sitting outside a mosque (cami = mosque) at night. Whatever you do make sure you have a backup plan because you can't trust the İETT website!
Keep an eye out for the Havataş bus since it'll leave from the same place as the city buses (but don't get on the similarly-named Havaş bus!). You should also be ready to abandon the whole public transit idea for a taxi at if you start running into problems. It's not actually that far, so it won't run up too much of a fare.