HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks
The aim of Few-Shot learning methods is to train models which can easily
adapt to previously unseen tasks, based on small amounts of data. One of the
most popular and elegant Few-Shot learning approaches is Model-Agnostic
Meta-Learning (MAML). The main idea behind this method is to learn the general
weights of the meta-model, which are further adapted to specific problems in a
small number of gradient steps. However, the model's main limitation lies in
the fact that the update procedure is realized by gradient-based optimisation.
In consequence, MAML cannot always modify weights to the essential level in one
or even a few gradient iterations. On the other hand, using many gradient steps
results in a complex and time-consuming optimization procedure, which is hard
to train in practice, and may lead to overfitting. In this paper, we propose
HyperMAML, a novel generalization of MAML, where the training of the update
procedure is also part of the model. Namely, in HyperMAML, instead of updating
the weights with gradient descent, we use for this purpose a trainable
Hypernetwork. Consequently, in this framework, the model can generate
significant updates whose range is not limited to a fixed number of gradient
steps. Experiments show that HyperMAML consistently outperforms MAML and
performs comparably to other state-of-the-art techniques in a number of
standard Few-Shot learning benchmarks.