Model selection based on validation corruptions.
Many previous works use the same model selection scheme as we used, i.e. selecting the optimal hyperparameters on the test corruptions [1]}, [2]}, [3]}. However, a proper validation-test split would lead to a more rigorous evaluation which we report in Table REF . We do a grid search based on the accuracy on the four validation corruptions from CIFAR-10-C. We see that for all adversarially trained models, the optimal \(\operatorname{\mathbf {\varepsilon }}\)  is the same as we reported in Table REF  except for the \(\operatorname{\mathbf {\ell }}_\infty \)  model on ImageNet-100 for which, however, the final test accuracy is unchanged. Overall, this is perhaps not too surprising as the validation corruptions are from the same category of corruptions and we optimize over a small one-dimensional grid so we are unlikely to overfit to the test corruptions in this way.
<TABLE>