(2) Regularisation and Constraint Optimisation. [1]}, [2]}, [3]} regularise the network update to alleviate catastrophic forgetting during learning new classes, and these approaches are often efficient and usually have little extra cost. For instance, [4]}, [5]} constrain the model optimisation such that the loss on past classes never increases. However, both works limit the space for the model optimisation, and hence they suffer from the inability of learning new classes.
