Over the years, I have visited Madagascar over 15 times, typically for 2-3 months at a time. I have been there during the January-March "rainy season" and I can honestly say it is a wonderful time to visit Madagascar, especially for flora and fauna.
The first thing to make clear is that it is not the rain of the rainy season that is a problem. Sure, you might get rained on more than at other times, but chances are there will be plenty of dry periods too (in the south you could well have no rain at all). The biggest issue with the rainy season is that most of Madagascar's roads are bad at the best of times, and a lot of them turn into impassable mudbaths during that season. Additionally, river levels rise, many river crossings become too dangerous, and some simple bridges can get washed away.
Consequently certain places are simply inaccessible at that time of year. There is no possibility at all, for example, to reach the Tsingy de Bemaraha National Park. And if, by some miracle, you did manage to get there, you would find that the park and all the hotels close for the rainy season anyway. So, if you do go to Madagascar in the rainy season, don't arrive with high expectations of reaching any difficult-to-access sites.
The other risk with the rainy season is that this is the time when cyclones are most likely to hit the island. Usually there's a few days' warning of incoming cyclones, so they are not so much a risk to your safety as just a potential disruption to your itinerary. Even during the rainy season, the likelihood is quite low of a cyclone hitting the specific part of the island where you are at the specific time you are there. So I wouldn't let this matter put you off, but I would advise having had some thoughts about a Plan B before your trip, just in case Plan A gets scuppered.
The most popular tourist route in Madagascar is from the capital city (in the centre) down to the southwest, following the Route Nationale 7. Much of this area, especially the further south you go, is not particularly rainy at any time of year - and this is also the part of the island least likely to be impacted by cyclones - so, if you are following this route, you might well hardly even notice that it's the rainy season at all.