Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning
Model-agnostic meta learning (MAML) is currently one of the dominating
approaches for few-shot meta-learning. Albeit its effectiveness, the
optimization of MAML can be challenging due to the innate bilevel problem
structure. Specifically, the loss landscape of MAML is much more complex with
possibly more saddle points and local minimizers than its empirical risk
minimization counterpart. To address this challenge, we leverage the recently
invented sharpness-aware minimization and develop a sharpness-aware MAML
approach that we term Sharp-MAML. We empirically demonstrate that Sharp-MAML
and its computation-efficient variant can outperform popular existing MAML
baselines (e.g., $+12\%$ accuracy on Mini-Imagenet). We complement the
empirical study with the convergence rate analysis and the generalization bound
of Sharp-MAML. To the best of our knowledge, this is the first empirical and
theoretical study on sharpness-aware minimization in the context of bilevel
learning. The code is available at https://github.com/mominabbass/Sharp-MAML.