Performance-critical machine learning models should be robust to input perturbations not seen during training. Adversarial training is a method for improving a model's robustness to some perturbations by including them in the training process, but this tends to exacerbate other vulnerabilities of the model. The adversarial training framework has the effect of translating the data with respect to the cost function, while weight decay has a scaling effect. Although weight decay could be considered a crude regularization technique, it appears superior to adversarial training as it remains stable over a broader range of regimes and reduces all generalization errors. Equipped with these abstractions, we provide key baseline results and methodology for characterizing robustness. The two approaches can be combined to yield one small model that demonstrates good robustness to several white-box attacks associated with different metrics.
tramer2018ensemble found that adversarial training on ImageNet often converges to a , leading to a kind of learned effect. Rather than becoming robust, it's conceivable that models may intentionally contort their decision boundaries s.t. adversarial examples are crafted that are easy to classify. tramer2018ensemble attempt to prevent this by transferring examples from an ensemble of pre-trained models, hence adversarial example generation from the model being learned. They claim improved robustness to some black-box transfer attacks, but at the expense of white-box performance. The ensemble method was omitted from the analyses of @cite_7 , as white-box robustness is strictly more difficult than in the black-box setting. The technique was shown to not hold in a more general black-box threat model using a query-efficient gradient estimator , and a gradient-free genetic algorithm .
@cite_7
2787708942
We identify obfuscated gradients as a phenomenon that leads to a false sense of security in defenses against adversarial examples. While defenses that cause obfuscated gradients appear to defeat optimization-based attacks, we find defenses relying on this effect can be circumvented. For each of the three types of obfuscated gradients we discover, we describe indicators of defenses exhibiting this effect and develop attack techniques to overcome it. In a case study, examining all defenses accepted to ICLR 2018, we find obfuscated gradients are a common occurrence, with 7 of 8 defenses relying on obfuscated gradients. Using our new attack techniques, we successfully circumvent all 7 of them.