The Consistency of Adversarial Training for Binary Classification
Robustness to adversarial perturbations is of paramount concern in modern
machine learning. One of the state-of-the-art methods for training robust
classifiers is adversarial training, which involves minimizing a supremum-based
surrogate risk. The statistical consistency of surrogate risks is well
understood in the context of standard machine learning, but not in the
adversarial setting. In this paper, we characterize which supremum-based
surrogates are consistent for distributions absolutely continuous with respect
to Lebesgue measure in binary classification. Furthermore, we obtain
quantitative bounds relating adversarial surrogate risks to the adversarial
classification risk. Lastly, we discuss implications for the $\cH$-consistency
of adversarial training.