Local vs global \(\operatorname{\mathbf {\ell }}_p\)  behavior.
Interestingly, adversarial training with worst-case perturbations bounded within a tiny \(\operatorname{\mathbf {\ell }}_2\)  ball leads to robustness significantly beyond this radius. Fig. REF  illustrates that common corruptions have an \(\operatorname{\mathbf {\ell }}_2\)  norm an order of magnitude larger than \(\operatorname{\mathbf {\varepsilon }}=0.1\)  used for \(\operatorname{\mathbf {\ell }}_2\)  adversarial training.
This is in contrast with adversarial robustness that does not significantly extend beyond the radius used for training [1]}.
Related to this, [2]} argue that for Gaussian noise improving the minimum distance to the decision boundary (e.g. via adversarial training) also leads to an improvement of the average distance. We have a similar mechanism at play for adversarial \(\operatorname{\mathbf {\ell }}_2\)  perturbations and common corruptions which may explain the generalization of adversarial training to large average-case perturbations. However, our setting is more complex compared to [2]} since at the training and test time we deal with different and diverse types of noise.
