These are called unmarked cars - sort of a compromise between a marked car and a fully undercover vehicle. The idea is that they're somewhat less easy to spot than a marked car, yet still have all the same equipment when it's needed.

They are very common, though the laws around their use may be evolving.

Yes, by law you do need to follow their instructions. If you are uncertain whether it's a genuine police car or an impostor, you can call 911 to verify.