Faster Optimization-Based Meta-Learning Adaptation Phase
Neural networks require a large amount of annotated data to learn.
Meta-learning algorithms propose a way to decrease the number of training
samples to only a few. One of the most prominent optimization-based
meta-learning algorithms is Model-Agnostic Meta-Learning (MAML). However, the
key procedure of adaptation to new tasks in MAML is quite slow. In this work we
propose an improvement to MAML meta-learning algorithm. We introduce Lambda
patterns by which we restrict which weight are updated in the network during
the adaptation phase. This makes it possible to skip certain gradient
computations. The fastest pattern is selected given an allowed quality
degradation threshold parameter. In certain cases, quality improvement is
possible by a careful pattern selection. The experiments conducted have shown
that via Lambda adaptation pattern selection, it is possible to significantly
improve the MAML method in the following areas: adaptation time has been
decreased by a factor of 3 with minimal accuracy loss; accuracy for one-step
adaptation has been substantially improved.