I recently got Electronic Travel Authorization (eTA) approved to travel to Canada by air. One thing I worry is that one alphabet letter in my passport number in the eTA approval email is not capitalized Should I reapply for the eTA in this case?
No, you don't have to re-apply.
The case of the letters in passport number does not matter.
This follows from the ICAO 9303-3 standard which has this about the characters used in the Machine Readable Zone:
Printed characters are restricted to those defined in Figure 4.
And 9303 part 4 which talks about travel documents mandates the passport number to be in the machine readable zone:
So if there would be two passports with the same characters except one of them in lowercase and the other in uppercase this would not be discernible for any machine readers which is clearly unacceptable. Perhaps you've seen when the border official scans your passport it's only the lower edge which gets swiped? That's the MRZ. Here's an example: