Recent studies show that widely used Deep neural networks (DNNs) are vulnerable to the carefully crafted adversarial examples.
 Many advanced algorithms have been proposed to generate adversarial examples by leveraging the L_p distance for penalizing perturbations.
 Different defense methods have also been explored to defend against such adversarial attacks. 
 While the effectiveness of L_p distance as a metric of perceptual quality remains an active research area, in this paper we will instead focus on a different type of perturbation, namely spatial transformation, as opposed to manipulating the pixel values directly as in prior works.
 Perturbations generated through spatial transformation could result in large L_p distance measures, but our extensive experiments show that such spatially transformed adversarial examples are perceptually realistic and more difficult to defend against with existing defense systems. This potentially provides a new direction in adversarial example generation and the design of corresponding defenses.
 We visualize the spatial transformation based perturbation for different examples and show that our technique
 can produce realistic adversarial examples with smooth image deformation.
 Finally, we visualize the attention of deep networks with different types of adversarial examples to better understand how these examples are interpreted. Deep neural networks (DNNs) have demonstrated their outstanding performance in different domains, ranging from image processing BID18 BID10 ), text analysis BID3 to speech recognition . Though deep networks have exhibited high performance for these tasks, recently they have been shown to be particularly vulnerable to adversarial perturbations added to the input images BID34 BID7 . These perturbed instances are called adversarial examples, which can lead to undesirable consequences in many practical applications based on DNNs. For example, adversarial examples can be used to subvert malware detection, fraud detection, or even potentially mislead autonomous navigation systems BID30 BID5 BID8 and therefore pose security risks when applied to security-related applications. A comprehensive study about adversarial examples is required to motivate effective defenses. Different methods have been proposed to generate adversarial examples such as fast gradient sign methods (FGSM) BID7 , which can produce adversarial instances rapidly, and optimization-based methods (C&W) BID1 , which search for adversarial examples with smaller magnitude of perturbation.One important criterion for adversarial examples is that the perturbed images should "look like" the original instances. The traditional attack strategies adopt L 2 (or other L p ) norm distance as a perceptual similarity metric to evaluate the distortion BID9 . However, this is not an ideal metric BID16 BID14 , as L 2 similarity is sensitive to lighting and viewpoint change of a pictured object. For instance, an image can be shifted by one pixel, which will lead to large L 2 distance, while the translated image actually appear "the same" to human perception. Motivated by this example, in this paper we aim to look for other types of adversarial examples and propose to create perceptually realistic examples by changing the positions of pixels instead of directly manipulating existing pixel values. This has been shown to better preserve the identity and structure of the original image BID44 . Thus, the proposed spatially transformed adversarial example optimization method (stAdv) can keep adversarial examples less distinguishable from real instances (such examples can be found in Figure 3 ).Various defense methods have also been proposed to defend against adversarial examples. Adversarial training based methods have so far achieved the most promising results BID7 BID38 BID28 . They have demonstrated the robustness of improved deep networks under certain constraints. However, the spatially transformed adversarial examples are generated through a rather different principle, whereby what is being minimized is the local geometric distortion rather than the L p pixel error between the adversarial and original instances. Thus, the previous adversarial training based defense method may appear less effective against this new attack given the fact that these examples generated by stAdv have never been seen before. This opens a new challenge about how to defend against such attacks, as well as other attacks that are not based on direct pixel value manipulation.We visualize the spatial deformation generated by stAdv; it is seen to be locally smooth and virtually imperceptible to the human eye. In addition, to better understand the properties of deep neural networks on different adversarial examples, we provide visualizations of the attention of the DNN given adversarial examples generated by different attack algorithms. We find that the spatial transformation based attack is more resilient across different defense models, including adversarially trained robust models.Our contributions are summarized as follows:• We propose to generate adversarial examples based on spatial transformation instead of direct manipulation of the pixel values, and we show realistic and effective adversarial examples on the MNIST, CIFAR-10, and ImageNet datasets.• We provide visualizations of optimized transformations and show that such geometric changes are small and locally smooth, leading to high perceptual quality.• We empirically show that, compared to other attacks, adversarial examples generated by stAdv are more difficult to detect with current defense systems.• Finally, we visualize the attention maps of deep networks on different adversarial examples and demonstrate that adversarial examples based on stAdv can more consistently mislead the adversarial trained robust deep networks compared to other existing attack methods. Different from the previous works that generate adversarial examples by directly manipulating pixel values, in this work we propose a new type of perturbation based on spatial transformation, which aims to preserve high perceptual quality for adversarial examples. We have shown that adversarial examples generated by stAdv are more difficult for humans to distinguish from original instances. We also analyze the attack success rate of these examples under existing defense methods and demonstrate they are harder to defend against, which opens new directions for developing more robust defense algorithms. Finally, we visualize the attention regions of DNNs on our adversarial examples to better understand this new attack. A MODEL ARCHITECTURES Here we evaluated adversarial examples generated by stAdv against the 3 × 3 average pooling restoration mechanism suggested in . TAB5 shows the classification accuracy of recovered images after performing 3 × 3 average pooling on different models. ImageNet-compatible. We use benign images from the DEV set from the NIPS 2017 targeted adversarial attack competition. 4 This competition provided a dataset compatible with ImageNet and containing target labels for a targeted attack. We generate targeted adversarial examples for the target inception_v3 model. In Figure 10 below, we show the original images on the left with the correct label, and we show adversarial examples generated by stAdv on the right with the target label.
We propose a new approach for generating adversarial examples based on spatial transformation, which produces perceptually realistic examples compared to existing attacks. 
{"src": "scitldr-AIC_train", "paper_id": "HyydRMZC-"}