We compare our proposed SDAF with a suite of state-of-the-art online continual learning algorithms:
First, EWC++ [1]} is an online version of elastic weight consolidation (EWC) [2]} that regularises the update of the crucial weights for the past classes, in which no replay buffer is used in such approach;
Experience replay (ER) [3]} and all the following baselines are built upon a replay buffer \(\mathcal {M}\) , where ER is among the simplest one which adopts reservior sampling [4]} for MemoryUpdate operations and uniform sampling for MemoryRetrieval as introduced in Algorithm REF  (noting that all the baselines described below also adopt the same sampling strategies for the MemoryUpdate and MemoryRetrieval operations unless otherwise specified);
Averaged gradient episodic memory (AGEM) [5]} utilises the samples in the replay buffer to constrain the update of the parameters;
Gradient-based sample selection (GSS) [6]} is a replay update method that improves MemoryUpdate operation by selecting samples with diversification of gradients to be stored in the replay memory;
Maximally interfered retrieval (MIR) [7]} provides an improved replay retrieval algorithm (related to MemoryRetrieval) that finds the samples impacted by the current batch for performing training iterations;
Adversarial shapley value experience replay (ASER) [8]} leverages Shapley value to determine the contribution of memory samples for learning performance, and develop the corresponding strategies to improve both the memory update and retrieval operations;
DualNet [9]} contains two components, called fast learner and slow learner. When it gets a new batch of data \(B^u_t\) , it first updates the slow learner by self-supervised learning (i.e. Barlow twins [10]}) on the samples from memory (\(B_M\) ), then it performs supervised learning to update both the fast learner and slow learner on \(B^u_t \bigcup B_M\) ;
Supervised contrastive replay (SCR) [11]} leverages the supervised contrastive learning to learn the representation for better continual learning performance, and it observed that the nearest class mean (NCM) classifier is better to deal with the recency bias than the softmax classifier;
SCL [12]} is not an online continual learning method initially. We borrow it from representation learning, with having all the settings to follow the SCR ones except the transformation module \(\mathcal {H}\)  as described in Figure REF .

DVC [13]} proposes the dual view consistency (which highly relates to contrastive learning but instead adopts mutual information in its objective) to fully utilises the single-pass data stream. Moreover, DVC provides an improved replay retrieval algorithm (related to MemoryRetrieval, cf. Algorithm REF  in the main manuscript) that selects the samples whose gradients generated in the network are most interfered by the new incoming samples, but such retrieval algorithm actually consumes 10 times forward operations (where the cost is estimated according to its hyperparameter setting) to select a single sample for replay, which is extremely costly for OCL thus being relatively impractical.
