A Survey on Machine Learning Adversarial Attacks <s> Privacy Attack Reprogramming Evasion <s> Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset. <s> BIB001 </s> A Survey on Machine Learning Adversarial Attacks <s> Privacy Attack Reprogramming Evasion <s> Deep CNNs are known to exhibit the following peculiarity: on the one hand they generalize extremely well to a test set, while on the other hand they are extremely sensitive to so-called adversarial perturbations. The extreme sensitivity of high performance CNNs to adversarial examples casts serious doubt that these networks are learning high level abstractions in the dataset. We are concerned with the following question: How can a deep CNN that does not learn any high level semantics of the dataset manage to generalize so well? The goal of this article is to measure the tendency of CNNs to learn surface statistical regularities of the dataset. To this end, we use Fourier filtering to construct datasets which share the exact same high level abstractions but exhibit qualitatively different surface statistical regularities. For the SVHN and CIFAR-10 datasets, we present two Fourier filtered variants: a low frequency variant and a randomly filtered variant. Each of the Fourier filtering schemes is tuned to preserve the recognizability of the objects. Our main finding is that CNNs exhibit a tendency to latch onto the Fourier image statistics of the training dataset, sometimes exhibiting up to a 28% generalization gap across the various test sets. Moreover, we observe that significantly increasing the depth of a network has a very marginal impact on closing the aforementioned generalization gap. Thus we provide quantitative evidence supporting the hypothesis that deep CNNs tend to learn surface statistical regularities in the dataset rather than higher-level abstract concepts. <s> BIB002
Evasion II. EVASION HE most common attack to ML system occurs during inference stage and is called evasion. It refers to designing an input, which seems normal for a human but is wrongly classified by ML models. A typical example is to change some pixels in a picture before uploading, so that image recognition A Survey on Machine Learning Adversarial Attacks Flávio Luis de Mello W T system computes a wrong classification. Figure 1 shows an adversarial example taken from Polyakov  can even fool humans. Szegedy et al. [9] provide a good mathematical formalization for the process of deceiving prediction models. Goodfellow et al. BIB001 followed Szegedy steps and produced interesting results as shown in Figure 2 illustration. An image was correctly classified as a panda, but when some noise is added to such image, the prediction model classifies the panda image as a gibbon with 99.3% of confidence. It is quite trivia to create imperceptible perturbation that completely fools Deep Neural Networks (DNN) as shown in Figure 2 . Jo and Bengio BIB002 suggests that Convolutional Neural Networks (CNN) are vulnerable to adversarial input attack because they tend to learn superficial dataset regularity instead of generalizing well and learning high-level representation that would be less susceptible to noise. 