I'm wondering if it's possible for passport holders of countries that do not recognise other countries to travel to that particular country. Israel is not recognised by the Malaysian passport, so would it be possible for a Malaysian (passport holder) to travel to Israel and just not have their passport stamp, or would Israel refuse the bearer entry/visa?
Israel allows Citizens of all countries to enter its borders, as long as they have an appropriate visa. There are countries for whose citizens Israel doesn't require a visa for a tourist stay of no more than 90 days. Malaysia and the other countries that don't recognize Israel and don't have diplomatic relations with Israel are not on that list so a Malaysian citizen would need to acquire a visa prior to their visit to Israel.
There is no Israeli representation in Malaysia, so a visa request would need to be filed at a representative at another country. Israel does have an embassy in Singapore. Or one might search the Israeli Foreign Affair Ministry site for more information.
Also it should be noted that while Israel may allow you to enter its borders, Malaysia has restrictions on visiting Israel. As noted in answers to other questions on this site, Israel allows for border control to not stamp a visitor's passport if they ask. This permits the the passport to have no record of the visit to Israel.
Every country has the sovereign right to determine who it will or will not allow into its territory. This is irrespective of the views of the country which issued the traveler's identity documents, or any markings thereon.
Therefore, the statement "Israel is not recognised by the Malaysian passport" is essentially meaningless. It is entirely up to the Israeli authorities to decide who they will allow into their territory on whatever basis they choose.
For specific information regarding visa and other entry requirements, I suggest contacting the nearest Israeli embassy. The following Israeli government website also contains visa information:
I have a Malaysian friend who did it. So yes it is possible. And he did not have any issues with the Malaysian authorities afterwards. I guess they couldn't know about his holiday trip. He took a flight to Jordan (a neighbor country) and from there drove to Israel with a car he rented. He even went to Gaza. I think he did that for the thrill, because he was excited to do something forbidden.