ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection
Class-incremental learning (CIL) learns a classification model with training
data of different classes arising progressively. Existing CIL either suffers
from serious accuracy loss due to catastrophic forgetting, or invades data
privacy by revisiting used exemplars. Inspired by linear learning formulations,
we propose an analytic class-incremental learning (ACIL) with absolute
memorization of past knowledge while avoiding breaching of data privacy (i.e.,
without storing historical data). The absolute memorization is demonstrated in
the sense that class-incremental learning using ACIL given present data would
give identical results to that from its joint-learning counterpart which
consumes both present and historical samples. This equality is theoretically
validated. Data privacy is ensured since no historical data are involved during
the learning process. Empirical validations demonstrate ACIL's competitive
accuracy performance with near-identical results for various incremental task
settings (e.g., 5-50 phases). This also allows ACIL to outperform the
state-of-the-art methods for large-phase scenarios (e.g., 25 and 50 phases).