Look-ahead meta-learning for continual learning
The continual learning problem involves training models with limited capacity to perform well on a set of an unknown number of sequentially arriving tasks. This setup can of- ten see a learning system undergo catastrophic forgetting, when learning a newly seen task causes interference on the learning progress of old tasks. While recent work has shown that meta-learning has the potential to reduce interference between old and new tasks, the current training procedures tend to be either slow or offline, and sensitive to many hyper-parameters. In this work, we propose Look-ahead MAML (La-MAML), a fast optimisation-based meta- learning algorithm for online-continual learning, aided by a small episodic memory. This is achieved by realising the equivalence of a multi-step MAML objective to a time-aware con- tinual learning objective adopted in prior work. The equivalence leads to the formulation of an intuitive algorithm that we call Continual-MAML (C-MAML), employing continual meta- learning to optimise a model to perform well across a series of changing data distributions. By additionally incorporating the modulation of per-parameter learning rates in La-MAML, our approach provides a more flexible and efficient way to mitigate catastrophic forgetting compared to conventional prior-based methods. This modulation also has connections to prior work on meta-descent, which we identify as an important direction of research to de- velop better optimizers for continual learning. In experiments conducted on real-world visual classification benchmarks, La-MAML achieves performance superior to other replay-based, prior-based and meta-learning based approaches for continual learning. We also demonstrate that it is robust, and more scalable than many recent state-of-the-art approaches.
Meta Learning Continual Learning Machine Learning E-learning Online learning Learning rate modulation