There is an FAQ on the federal ministery of the Interior about residence permits. Unfortunately, one can not simply switch to english, and I didn't find it on the english site.
It states:
Muss ich immer meinen Pass oder Passersatz mit mir führen?
Nein, aber Sie müssen auf Verlangen den Ausländer- und Polizeibehörden Ihren Pass bzw. Passersatz und Ihren Aufenthaltstitel vorlegen.
[...]
- Wenn Sie keinen Pass besitzen, diesen auch nicht zumutbar erhalten können oder Ihren Pass vorübergehend einer deutschen Behörde überlassen haben, müssen Sie umgehend einen Ausweisersatz beantragen.
which translates to
Do I always have to have my passport or replacement with me?
No, but upon request of police or aliens authorites, you have to present your passport or replacement and your residence permit
[...]
- If you do not have an ID, you can not get it with reasonable effort, or have passed it temporary to a German authority, you must apply for a replacement passport.
So, it boils down to: You don't need to have your passport with you, but if police asks, you have to present it...
ID card and passport are the only official documents to confirm your identity, the residence permit is only a supplement.
There is no general passport control for flights within the Schengen area and you can often board an aircraft without showing your passport anywhere. Airline staff may be fine with anything looking official, like residence permit or drivers license. But it is still possible that police asks you for your passport. Maybe, you get away with your story, maybe, not.
(Within this context, it is interesting how a consulate can take your passport, leaving you without any official ID. As German, you can easily get a replacement as requested by the last paragraph of the quote, but for others... hmmm)
Sorry, I don't have a solution for your particular problem, this is just the law.
