Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile
Recent years have seen a surge of interest in meta-learning techniques for
tackling the few-shot learning (FSL) problem. However, the meta-learner is
prone to overfitting since there are only a few available samples, which can be
identified as sampling noise on a clean dataset. Moreover, when handling the
data with noisy labels, the meta-learner could be extremely sensitive to label
noise on a corrupted dataset. To address these two challenges, we present
Eigen-Reptile (ER) that updates the meta-parameters with the main direction of
historical task-specific parameters to alleviate sampling and label noise.
Specifically, the main direction is computed in a fast way, where the scale of
the calculated matrix is related to the number of gradient steps instead of the
number of parameters. Furthermore, to obtain a more accurate main direction for
Eigen-Reptile in the presence of many noisy labels, we further propose
Introspective Self-paced Learning (ISPL). We have theoretically and
experimentally demonstrated the soundness and effectiveness of the proposed
Eigen-Reptile and ISPL. Particularly, our experiments on different tasks show
that the proposed method is able to outperform or achieve highly competitive
performance compared with other gradient-based methods with or without noisy
labels. The code and data for the proposed method are provided for research
purposes https://github.com/Anfeather/Eigen-Reptile.