Understanding Benign Overfitting in Nested Meta Learning
Meta learning has demonstrated tremendous success in few-shot learning with
limited supervised data. In those settings, the meta model is usually
overparameterized. While the conventional statistical learning theory suggests
that overparameterized models tend to overfit, empirical evidence reveals that
overparameterized meta learning methods still work well -- a phenomenon often
called ``benign overfitting.'' To understand this phenomenon, we focus on the
meta learning settings with a challenging nested structure that we term the
nested meta learning, and analyze its generalization performance under an
overparameterized meta learning model. While our analysis uses the relatively
tractable linear models, our theory contributes to understanding the delicate
interplay among data heterogeneity, model adaptation and benign overfitting in
nested meta learning tasks. We corroborate our theoretical claims through
numerical simulations.