Another possible reason to keep your old passports is as a continuous record of identity.
A while ago I knew someone that had trouble with passports twice - lost one, I think, and something else. In any case, when he was going to renew it the next time, there came up another problem with his birth certificate - that no one had mentioned before, not even with other passports... possibly just the extra scrutiny this time. Since there wasn't time before travel to have another copy mailed to him, he brought along all the additional proofs of identity he could, hoping to get his passport renewed in time.
This included every expired passport he had - as a continuous record, of who he is and proof that his identity was real. He had them going all the way back to childhood, because of overseas relatives. Just like an expired passport can still be used as proof of identity in some places, a record of them shows that someone's identity has history.
I don't know if the passports were the deciding factor, or if the other documents he had were enough, but they took them and looked them over (instead of being confused at him, which suggests that they can indeed be used as this sort of continuous record), and they did issue him his passport, so I guess it was helpful to have them.
I suppose it's pretty unlikely that someone would need to prove their identity in this way, or to use them to show a continual proof of identity - but it might happen, however rare, and it is probably not a bad thing to keep the old passports anyway. If something happened to your passport (or even most-recently-expired one), then having an even older one might someday be an extra kind of backup, as proof of citizenship, as one more piece of bureaucratic proof, or anything else you want it for.