On the other hand, data augmentation has been a boon to practitioners, aiding in state-of-the-art performance on a variety of tasks [1]}, [2]}. Data augmentation can be used in many regimes, including settings where data is sparse, to improve generalization [3]}. Simple augmentations include random crops or horizontal flips. Recently, more sophisticated augmentation schemes have emerged that improve model performance: mixup takes pairwise convex combinations of randomly sampled training data and uses the corresponding convex combinations of labels. Not only does this prevent memorization of corrupt labels and provide robustness to adversarial examples, but it has also been shown to improve generalization [1]}. Another augmentation technique, cutout, randomly erases patches of training data [5]}, whereas CutMix instead combines pairwise randomly sampled training data by taking random patches from one image and overlaying these patches onto other images [6]}. The labels are then mixed proportionally to the area of these patches. CutMix enhances model robustness, achieves better test accuracy, and improves localization ability by encouraging the network to correctly classify images from a partial view. Finally, MaxUp applies a set of data augmentation techniques (basic or complex) to the training data and chooses the augmentation method and parameters that achieve the worst model performance of all the techniques [2]}. By training against the most “difficult" data augmentation, MaxUp is able to improve generalization and, in some cases, adversarial robustness.
