Catastrophic overfitting is a bug but also a feature
Despite clear computational advantages in building robust neural networks,
adversarial training (AT) using single-step methods is unstable as it suffers
from catastrophic overfitting (CO): Networks gain non-trivial robustness during
the first stages of adversarial training, but suddenly reach a breaking point
where they quickly lose all robustness in just a few iterations. Although some
works have succeeded at preventing CO, the different mechanisms that lead to
this remarkable failure mode are still poorly understood. In this work,
however, we find that the interplay between the structure of the data and the
dynamics of AT plays a fundamental role in CO. Specifically, through active
interventions on typical datasets of natural images, we establish a causal link
between the structure of the data and the onset of CO in single-step AT
methods. This new perspective provides important insights into the mechanisms
that lead to CO and paves the way towards a better understanding of the general
dynamics of robust model construction. The code to reproduce the experiments of
this paper can be found at https://github.com/gortizji/co_features .