Uncertainty guided Curriculum Learning via Infinitesimal Jackknife
A key challenge in curriculum learning (CL) is to design reasonable metrics for scoring sample difficulty. Current data driven CL algorithms need to learn extra parameters for the sample scoring task. In contrast, we propose a novel approach, called uncertainty guided curriculum learning (UGCL) where the sample difficulty is scored according to both the uncertainty in model's decision making (epistemic) and the uncertainty inherent in data (aleatoric) Besides, we show a novel scoring metric based on signal tonoise ratio (SNR) which combines the uncertainty and predicted probability of the target class (measuring the difficulty for the model to make a correct prediction) together to evaluate sample difficulty simultaneously. Moreover, in order to avoid burdensome learning on Bayesian neural network, we apply the infinitesimal jackknife (IJ) to quantify uncertainty in model weights through a pseudo ensemble approach based on Fisher information matrix. Experiments on various image and text datasets empirically verify the superiority of UGCL over the state of the art CL baselines. The source code of our algorithm is available in https:/github.com/RyanWangZf/ Uncertainty Curriculum Learning.