Tip: Only use search engines to find out who flies between the city pairs and use the airline website for actual booking, you will have a much better selection.
I happen to know Qatar has free-to-cancel tickets and on a random date it was actually cheapest on Kayak so I checked Mumbai-Johannesburg on qatarairways.com and the fares / cancel fees in Economy Promo | Saver | Value | Flexi are
19016 / 2340 | 21846/ 2005 | 30803 / 1340 | 45724 / 0 . So Economy Flexi is free to cancel but the 2.5 times cheaper Promo is not horrible to cancel either.
Etihad, another very cheap option has a similar price range of 20223-42033 INR but the cancel fees are much steeper, 7000-1000 -- even the most expensive ticket costs to cancel.
Since I searched kayak and it indicated Kenyan Airways as a cheap option as well, I first checked whether it's safe and it passed IATA IOSA so it's OK then checked the prices and I found three sources on their website giving contradictory information about whether the Economy Super Flex is free to refund or not. A call would be required.