RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustness
We show that the effectiveness of the well celebrated Mixup [Zhang et al.,
2018] can be further improved if instead of using it as the sole learning
objective, it is utilized as an additional regularizer to the standard
cross-entropy loss. This simple change not only provides much improved accuracy
but also significantly improves the quality of the predictive uncertainty
estimation of Mixup in most cases under various forms of covariate shifts and
out-of-distribution detection experiments. In fact, we observe that Mixup
yields much degraded performance on detecting out-of-distribution samples
possibly, as we show empirically, because of its tendency to learn models that
exhibit high-entropy throughout; making it difficult to differentiate
in-distribution samples from out-distribution ones. To show the efficacy of our
approach (RegMixup), we provide thorough analyses and experiments on vision
datasets (ImageNet & CIFAR-10/100) and compare it with a suite of recent
approaches for reliable uncertainty estimation.