As Ilmari commented, the recent NY Times travel blog post by "frugaltraveler" is very good. It even mentions my favorite bar, Pub Magneetti. I like his other recommendations too: free concerts & somewhat hippie atmosphere at Alppipuisto; public sauna in Kallio; berry-picking).

Some more ideas below. Many of these may not be "only in Helsinki" things, but stuff I love about this place anyway. (I've lived in Helsinki for almost 14 years.) The sea and nature spots are never far even in central Helsinki; for me a big part of the spirit is related to that.

On a warm summer day, head to Suomenlinna for a picnic [map]. It's truly a unique place for this, and a Unesco World Heritage site too. Buy some strawberries from the Market Square before boarding the ferry. Maybe hit the supermarket on the main island for some food, beers or ciders. (What locals do anyway.) When leaving, please take your trash with you. :-)

^{Photo: Michal Pise, Wikimedia Commons}

Go to Hietsu (Hietaniemi), the most famous sandy beach in Helsinki, a walking distance from centre [map]. The beach itself (or the cool water) may not impress you much, but on hot days it's a really lively place with a mostly youthful crowd. Besides sunbathing, you could join some of the many groups playing beach volley.

...Or if it's Sunday, come to the park very close to Hietsu at Hesperiankatu & Mechelininkatu [map] to play ultimate frisbee with local & visiting couchsurfers, starting 3pm. Then, continue to Töölönlahti: at the amphitheatre behind the National Opera [map], there's a (free) salsa party on summer Sundays starting 6pm. Whether you dance or just hang around, the atmosphere is lovely.

^{Salsa at the Opera. Photo by me.}

On the opposite side of Töölönlahti, overlooking the bay, there are some lovely old villas. Go to Café Sininen huvila ("blue villa") to have a coffee and read a book perhaps. [map]

Besides Hietsu, good places for swimming and spending a summer day are the city's two public open-air pools: the Swimming stadium [map] and Kumpulan maauimala [map] (admission 3.60 € at both places). To me, feel somehow "traditionally Finnish", in a good way. There's a sauna too, of course. The one in Kumpula is farther from centre, but located in a charming residential area with wooden houses. (Only downside is that it can get crowded sometimes.)

Climb to the rocks in Vallila [map] to watch the sunset and views over the beautiful old wooden buildings of this working-class neighborhood turned hip. (Combine this with visiting Pub Magneetti at Mäkelänkatu 20.) Or for a sunset in a more bourgeois environs, try Munkkiniemi on the western side of Helsinki peninsula, just where tram 4 terminates [map]. There's a lovely (and expensive) café with a terrace there, or you can just hang around on the seashore.

^{Sunset at Munkkiniemi, August 2012. Photo by me.}

For a proper nature retreat, you could do like the NYT blogger and head to Sipoonkorpi, but it would be somewhat easier to reach Uutela nature park in eastern Helsinki by the sea (take the metro to Vuosaari, then walk) [map]. Nuuksio in Espoo is also great for this though a little farther away [map].

If it's your cup of tea, getting drunk—and getting to know locals in the process—is obviously one of the most popular ways to feel the spirit of Helsinki. Tour the (relatively) cheap bars in Kallio, for example, and head to city centre for later-night action. If it's Thursday, go to "Kalle" (Kaarle XII) which for some reason is packed (only) that day (go before 10pm to avoid a long queue). (Alcohol is generally expensive in bars/clubs; what Finns commonly do is pre-drink at a house party or a park or something.)

Follow the events organised by We Love Helsinki collective. All kinds of urban culture, parties, bike trips, artsy events. Really fun stuff. (Find someone to decipher the Finnish for you.)

NB: almost all of the above is quite summer-specific. But then Helsinki is at its best in summer and that's when most tourists come here. In winter, you could try ice skating at Rautatientori, the square next to central railway station (admission 5€ + skate rental 5€). (I'll edit this later if I forgot something.)