To address this issue, [1]} propose using a small memory to store the learnt examples. When learning new classes, the model is retrained/updated by using not only the recently received samples that belong to new classes but the stored examples from old classes, in which such a strategy attempts to maintain the accuracy for both the old and new classes.
Another widely-adopted idea comes from [2]}, where they propose to regularise the model learning by constraining the update of important parameters in order to alleviate the catastrophic forgetting. However, limiting the model update space deteriorates the learning ability of the model.
Recently, the introduction of learning a generalisable representation [3]}, [4]}, [5]}, [6]} brings another break.
Basically, learning the generalisable representation aims to not only distinguish between the learnt/seen classes but also have higher intrinsic dimension such that the features from unseen classes are more likely to be distributed away from the seen ones (cf. Figure REF ), i.e. the feature representations are richer and more reusable/transferable for the unseen classes in which the model requires relatively minor adjustments to achieve high accuracy for the new classes.
