We start by showing in Fig. REF  the common corruption accuracy of \(\operatorname{\mathbf {\ell }}_\infty \)  adversarially trained models as it is the most widely studied setting [1]} and has been reported multiple times in common corruption literature [2]}, [3]}, [4]}.
Since we are interested primarily in small-\(\operatorname{\mathbf {\varepsilon }}\)  adversarial training, we rely throughout the paper on FGM/FGSM for \(\operatorname{\mathbf {\ell }}_2\) /\(\operatorname{\mathbf {\ell }}_\infty \)  norms respectively to solve the inner maximization problem (REF ) which only leads to a \(2\times \)  computational overhead.
Note however that we exceptionally use PGD with 10 steps for \(\operatorname{\mathbf {\varepsilon }}\in \lbrace {8}{255}, {10}{255}\rbrace \)  to prevent catastrophic overfitting
and allow a direct comparison with previous works.
We observe that for the small-\(\operatorname{\mathbf {\varepsilon }}\)  regime around \(\operatorname{\mathbf {\varepsilon }}={1}{255}\) , we get a significant improvement in corruption accuracy: 74.5% accuracy is achieved with standard training, 82.7% with adversarial training using \(\operatorname{\mathbf {\varepsilon }}={1}{255}\) , and 73.8% using the standardly reported threshold \(\operatorname{\mathbf {\varepsilon }}_\infty ={8}{255}\) .The exact numbers differ from [3]} since we use ResNet-18 instead of WRN-28-10 and different hyperparameters.
The reason is that the tradeoff between robustness and accuracy [6]} has to be carefully balanced—if the standard accuracy drops for higher \(\operatorname{\mathbf {\varepsilon }}\) , the corruption accuracy also deteriorates. Thus, selecting the most robust \(\operatorname{\mathbf {\ell }}_p\) -model does not lead to the optimal performance on common corruptions.
Alternatively, one can also balance this tradeoff by mixing clean and adversarial samples, but it overall leads to similar results (see App.  for details), so we focus on adversarial training with 100% adversarial samples for the rest of the paper.
<FIGURE>