Progressive Latent Replay for efficient Generative Rehearsal
We introduce a new method for internal replay that modulates the frequency of
rehearsal based on the depth of the network. While replay strategies mitigate
the effects of catastrophic forgetting in neural networks, recent works on
generative replay show that performing the rehearsal only on the deeper layers
of the network improves the performance in continual learning. However, the
generative approach introduces additional computational overhead, limiting its
applications. Motivated by the observation that earlier layers of neural
networks forget less abruptly, we propose to update network layers with varying
frequency using intermediate-level features during replay. This reduces the
computational burden by omitting computations for both deeper layers of the
generator and earlier layers of the main model. We name our method Progressive
Latent Replay and show that it outperforms Internal Replay while using
significantly fewer resources.