I would suggest that you have already experienced the consequences: You had an opportunity to become a US permanent resident, and if you had done so you most likely could have been naturalized and had a US passport by now.
Choosing to return to the Philippines, you must have had very good reasons for doing so. These reasons indicate strong ties to your country, and that is one of the important things that you must demonstrate in order to get a UK visa.
So, I suppose you could say that it counts against you because you can't enter visa-free as a US citizen, but it doesn't count against you because it means you have strong ties to your country which can demonstrate that you are not likely to remain in the UK illegally.
Think about the reasons which keep you living where you are. Family, work, property ownership... these are much more important to your visa application than your old US immigrant visa.