[1]} have shown that using adversarial transformations - which as already mentioned is related to promoting invariance - can give competitive results with respect to other automatic augmentation methods in image classification. However, [2]} have since evidenced the importance of two factors: the implicit learning curricula and the suboptimality of the adversarial used by [1]}, which mitigate the biases introduced by worst-case transformations. Furthermore, [2]} report that an explicit cyclic curricula in which augmentations are mild at first, then get harder as training progresses, and finally revert to milder augmentations at the end of training, performs better empirically. We note some interesting commonalities with our approach and experimental results. First, the dynamics of our primal-dual algorithm resemble the aforementioned heuristically defined curricula. Second, the suboptimality with respect to worst case perturbations can be related to the smoothed approximation used in our approach.
