Set-based Meta-Interpolation for Few-Task Meta-Learning
Meta-learning approaches enable machine learning systems to adapt to new
tasks given few examples by leveraging knowledge from related tasks. However, a
large number of meta-training tasks are still required for generalization to
unseen tasks during meta-testing, which introduces a critical bottleneck for
real-world problems that come with only few tasks, due to various reasons
including the difficulty and cost of constructing tasks. Recently, several task
augmentation methods have been proposed to tackle this issue using
domain-specific knowledge to design augmentation techniques to densify the
meta-training task distribution. However, such reliance on domain-specific
knowledge renders these methods inapplicable to other domains. While Manifold
Mixup based task augmentation methods are domain-agnostic, we empirically find
them ineffective on non-image domains. To tackle these limitations, we propose
a novel domain-agnostic task augmentation method, Meta-Interpolation, which
utilizes expressive neural set functions to densify the meta-training task
distribution using bilevel optimization. We empirically validate the efficacy
of Meta-Interpolation on eight datasets spanning across various domains such as
image classification, molecule property prediction, text classification and
speech recognition. Experimentally, we show that Meta-Interpolation
consistently outperforms all the relevant baselines. Theoretically, we prove
that task interpolation with the set function regularizes the meta-learner to
improve generalization.