I was wondering if I need a passport or I can simply use my EU ID card to enter and exit Albania?
Yes, you can use your ID card instead of your passport. The November 2018 list (pdf) shows that all European Union countries are among those whose citizens are permitted to enter Albania with a national ID card.
It should be noted that "national ID card" generally means an identification card issued by a national government that identifies the holder as a national of that country. In particular, a driver's license does not suffice.