It really depends on what you're interested in. I suppose that you're looking for art museums like you mentioned in your example.
Therefore I really can recommend this list of art museums in Germany:
I don't know all of them, but I visited and can recommend:
- Akademie der Künste in Berlin
- Mies van der Rohr Haus in Berlin
- Alte Pinakothek in Munich
- Architekturmuseum der Technischen Universität in Munich
- Museum for Applied Arts in Cologne
If you really want to just visit the 2 or 3 most important sights in whole Germany, I would recommend first spend some days in Berlin, the either go to the area around Cologne and then to Munich or over Dresden to Munich.
In Berlin, there are really a lot of interesting sights, especially a lot of museums (e.g. the Museumsinsel, a island that is full of museums) and naturally a lot of places that remember the division of Germany (e.g. Checkpoint Charley) or the Holocaust. In my opinion also the TV tower is really worth a visit. You can also check out the Brandenbourg Gate, the Column of Victory (Siegessäule) or the Bundestag.
In Munich there is also a lot to do. You have the choice between some cultural activities like museums, or you can spend your time in the various beer gardens, especially the Hofbräuhaus are known worldwide. At the moment there is also the Oktoberfest that attracts many visitors. I personally would also visit the old Olympic Stadium with its famous roof designed by Günther Behnisch. There you can also climb a tower that will provides you with a great look over the city.