I have a study visa for 10 months (duration of the course) and entered EU using this visa. I want to travel right after my course on a Schengen visa. Is this possible or do I need to exit Europe for 180 days to do so?
Stay within the Schengen area under a long term visa (like your 10 month study visa) does not count for the purposes of the 90 days in 180 calculation. See here, and in particular the user's guide for the short stay calculator.
Periods of stay authorised under a residence permit or a long-stay visa shall not be taken into account in the calculation of the duration of stay on the territory of the Member States
Assuming you a citizen of a Schengen visa-waiver country, you are free to leave Schengen at the end of your long stay visa, and then immediately seek readmission for a short stay (although it might not be advisable to try and reenter the country which issued your long-stay visa, since they may suspect you are trying to continue residence). If you are not a citizen of a Schengen visa-waiver country, you would of course need to apply for a short stay visa before reentering.