Background on adversarial training.
Let \(\operatorname{\mathbf {\ell }}(x,y;\theta )\)  denote the loss of a classifier parametrized by \(\theta \in \mathbb {R}^m \)  on the sample \((x,y)\sim D\)  where \(D\)  is the data distribution.
Previous works [1]}, [2]} formalized the goal of training adversarially robust models as the following optimization problem:
\(\mathop {\rm min}\limits _{\theta } \operatorname{\mathbb {E}}_{(x, y) \sim D} \big [ \mathop {\rm max}\limits _{\delta \in \Delta } \operatorname{\mathbf {\ell }}(x+\delta , y; \theta ) \big ].\) 
