I want to know if I require a transit visa
You will need a EVW (Electronic Visa Waiver) or a transit/stopover Visa see https://www.visitsaudi.com/en/saudi-stopover?lang=en and [UK Foreign office]
EVW would be the safest route to go since the airline for the first leg will be checking your ability to enter Saudi Arabia and may deny you boarding if you don't have proper documentation.
and have to exit out or can I internally go to the RUH-ISB gates as I have no checked in luggage.
Unknow. This depends on the specific airlines, which gates they are using and the layout of the airport. It also depends on how you get the boarding passes for the second flight. You may have to go to the check in counter at RUH is any type of doc check is required for check-in.
The stop over at RUH is only 1hr30mins and ... says 'self transfer
Duplicating my comment: Personally I would not risk an international self transfer with only 90 minutes. If you miss the connection, you will have to buy a same day or next day ticket which can be VERY expensive.