As I understand it, your brother stayed for 2 years after he stopped studying? He can apply for a visa, but his chances of getting it might not be very good.
People on F1 are almost always admitted on "D/S" on their I-94, which means they do not automatically start accruing "unlawful presence" on any date. Assuming he didn't apply to USCIS for some benefit and was denied for being out of status, and he didn't receive a final order of deportation, he didn't accrue any "unlawful presence" before he left, and he does not have a 3-year/10-year unlawful presence ban. (There was a policy change to consider F1 to accrue "unlawful presence" whenever out of status after August 9, 2018, but that doesn't matter here because 1) he left before that, and 2) the policy change has been blocked by a nationwide injunction anyway.)
But just because he doesn't have a ban doesn't mean he will get a visa. Visa officers have wide discretion to deny most types of nonimmigrant visas, including visitor visas, for "failure to overcome the presumption of immigrant intent" if there is anything they don't like about the person's case. And a past history of overstay is definitely a negative factor.