There are other approaches that are unable to categorise into the above three branches.
First, knowledge distillation [1]} keeps an old model as a teacher to preserve the learnt knowledge [2]}, [3]}, [4]}.
Second, the expansion-based online continual algorithm dynamically expands the network capacity upon the arrival of new classes [5]}, [6]}.
We do not take these approaches in account here, as they need additional resources for computation (e.g. much more additional memory for storing the old model to perform knowledge distillation; and continuously growing model size for the expansion-based methods in which it means that the requirement of memory space also keeps increasing), and we only consider the methods with similar computational cost as our baselines to make comparison.
