You won't find much, except bars and such, open at that hour.
You can of course visit many noteworthy buildings, if you are content to just see them from the outside. This includes Christiansborg, Amalienborg, Marble Church, Vor Frelsers Kirke and many others. There are also interesting landmarks such as Nyhavn and the statue of the Little Mermaid. I'd recommend reading up a bit so you'll have some context when walking around.
I'd also highly recommend a stroll down Strøget. It is a walking street that stretches from Nyhavn and almost to the main train station. On a Friday night it will be full of life and is certainly a part of Copenhagen's culture, even if it is not highbrow culture.