It depends, but most likely it is a very bad idea. First of all you have to realize that there is no "refugee status in the EU". Instead, you have been given the refugee status by some specific country (which itself is a member of the European Union). Many laws regarding refugees are defined by national laws and on top of that are some EU laws (especially the Dublin Regulation), so whether what you are doing is a bad idea depends on the country which issued your refugee travel documents.
However, it is most likely a very bad idea, because internationally it is well established that a refugee status can be revoked when you return to the protection of the country you were seeking refugee from. But again, this has to be established in the individual laws of the country, so to say for sure we would have to know which country you currently live in.
I can explain the example for Germany, the most populous country within the EU. I would suspect most (if not all) other European countries having similar laws. In §72 AsylG Absatz 1 Satz 1 it says:
Die Anerkennung als Asylberechtigter und die Zuerkennung der Flüchtlingseigenschaft erlöschen, wenn der Ausländer sich freiwillig durch Annahme oder Erneuerung eines Nationalpasses oder durch sonstige Handlungen erneut dem Schutz des Staates, dessen Staatsangehörigkeit er besitzt, unterstellt
which roughly translates to
The refugee status is void, if the foreigner voluntarily accepts or renews their passport or in any other way seeks the protection of the country of origin.
So to be clear: If you received your refugee status in Germany the action you proposed would definitely result in losing your refugee status. To be absolutely clear, this will certainly happen as the law explicitly does not define any special circumstances or "may" clauses - It is the only possible legal outcome.
I would also like to address your initial statement that you can't travel to this particular country because it does not recognize your travel document. However, all countries which ratified the 1951 or 1967 Convention should in fact recognize your documents. Wikipedia has a nice map showing these countries, with only the gray states not having joined the Conventions: https://en.wikipedia.org/wiki/Convention_relating_to_the_Status_of_Refugees#/media/File:Refugeeconvention.PNG
And even these states could still recognize your documents. The refugee travel documents are internationally widely accepted. Hence, I am a bit unconvinced that this country really does not recognize the document at all. However, it might be a lot more cumbersome (e.g. with your other passport you could travel visa free).
If you really want to travel to this country you should consult a lawyer how you could achieve that. But the way you proposed is not a good idea, given its possible drastic consequences.