Adaptive Regularization for Adversarial Training
Adversarial training, which is to enhance robustness against adversarial
attacks, has received much attention because it is easy to generate
human-imperceptible perturbations of data to deceive a given deep neural
network. In this paper, we propose a new adversarial training algorithm that is
theoretically well motivated and empirically superior to other existing
algorithms. A novel feature of the proposed algorithm is to use a data-adaptive
regularization for robustifying a prediction model. We apply more
regularization to data which are more vulnerable to adversarial attacks and
vice versa. Even though the idea of data-adaptive regularization is not new,
our data-adaptive regularization has a firm theoretical base of reducing an
upper bound of the robust risk. Numerical experiments illustrate that our
proposed algorithm improves the generalization (accuracy on clean samples) and
robustness (accuracy on adversarial attacks) simultaneously to achieve the
state-of-the-art performance.