For men, the only place you have to worry about what you are wearing is inside a mosque, other than that you can wear whatever you want to. This does not apply to women, though.
However in January or February I think it is going to be a little cold for shorts, t-shirts and flip flops.
In Aswan, Abo Simbel and Luxor it's only at night when it gets cold just as any desert, which can be very cold sometimes, but the rest of the day you're fine with a t-shirt.
Cairo may be a very little bit warm a few hours a day and Alexandria has crazy weather that changes every hour with wet conditions and low temperature.
Anyway here are the average temperatures for the cities you are visiting:
January(Lo/Hi): 8-18c / February: 9-20c
January: 7-17c / February: 8-18c
January: 7-24c / February: 9-26c
January: 5-23c / February: 6-25c
January: 8-23c / February: 9-23c
And these are just average temperatures inside the cities, as soon as you are at the edge of any city you can feel the temperature drop to much lower as you go more into the desert.
only 50km away from down town Cairo (Cairo is Huge) the temperature was 3c at 2 am. while as soon as I entered Cairo the temperature was 18c at 2:25 am.
and I’ve noticed an increase in extreme conditions, the weather doesn’t match with forecasts as it used to. sometimes it changes suddenly high or low, and in multiple recent occasions it rained ice in Cairo and Alexandria while the temperature in Cairo was 17c, I don’t know whether to call it a Hail storm or not because there wasn’t any storm, just big chunks of ice falling from the sky.
Also rain at strange times of the year and much more than used to.