That, or other extrinsic evidence. The easiest would be inconsistent deposition testimony (some lawyers are good at making you say what they want -- don't underestimate them), but another option would be evidence of your having done something to manifest your knowledge of the thing you claim to have forgotten soon before or after the deposition (like, e.g., sending an email).

And while perjury may be a stretch in most cases, there is also the matter of the jury's opinion of you as a witness (assuming that the case eventually goes to trial and you get confronted with your own deposition transcripts). If they think your memory seems a tad selective, then they're apt to assume that any fact you've "forgotten" is the worst possible fact for you. There are also concerns like civil/criminal contempt and other sanctions against your side in the litigation.

There is also the small matter of morality. Many of us think that you shouldn't lie in a deposition even if you can get away with it.