Without loss of generality, here we summarise a generic algorithmic procedure for the class-incremental online continual learning methods with the experience replay memory in Algorithm REF . Basically, when the machine learning agent receives a data batch \(B_{t}^u\)  during the training stage \(t\) , it will use the data from \(B_{t}^u\)  (belonging to the new classes) as well as the samples from the experience replay memory (mostly belonging to the old classes) to train itself for \(I\)  iterations (for simplicity, we assume that \(I\)  is a constant). Please note that, we just train the same sample for \(I\)  times which does not violate the one epoch setting. In particular, at each iteration, the agent utilises the data \(B_{t}^u \cup B_M\)  to perform the training where \(B_M\)  denotes the \(m\)  samples retrieved from the experience replay memory. After \(I\)  iterations of training based on the batch \(B_{t}^u\)  and the memory, the MemoryUpdate operation is performed to replace some exemplars in the memory \(\mathcal {M}\)  with the ones sampled from \(B_{t}^u\) ,
where the reservoir sampling algorithm [1]} is adopted for such a MemoryUpdate operation in our work following the practise in [2]}.
<FIGURE>