Machine learning models including traditional models and neural networks can be easily fooled by adversarial examples which are generated from the natural examples with small perturbations.   This poses a critical challenge to machine learning security, and impedes the wide application of machine learning in many important domains such as computer vision and malware detection.   Unfortunately, even state-of-the-art defense approaches such as adversarial training and defensive distillation still suffer from major limitations and can be circumvented.   From a unique angle, we propose to investigate two important research questions in this paper: Are adversarial examples distinguishable from natural examples?   Are adversarial examples generated by different methods distinguishable from each other?   These two questions concern the distinguishability of adversarial examples.   Answering them will potentially lead to a simple yet effective approach, termed as defensive distinction in this paper under the formulation of multi-label classification, for protecting against adversarial examples.   We design and perform experiments using the MNIST dataset to investigate these two questions, and obtain highly positive results demonstrating the strong distinguishability of adversarial examples.   We recommend that this unique defensive distinction approach should be seriously considered to complement other defense approaches. Machine learning models including SVMs BID0 and especially deep neural networks BID17 can be easily fooled by adversarial examples which are generated from the natural examples with small perturbations. Quite often, both machine learning models and humans can classify the natural examples such as the images of pandas with high accuracy, and humans can still classify the adversarial examples as pandas with high accuracy because the small perturbations are imperceptible; however, machine learning models are fooled to misclassify adversarial examples as some targets such as gibbons BID4 desired by attackers.This intriguing property or vulnerability of machine learning models poses a critical challenge to machine learning security, and it impedes the wide application of machine learning in many important domains such as computer vision (e.g., for self driving cars) and even in malware detection BID0 BID19 . Furthermore, the discovery of new and powerful adversarial example generation methods such as BID17 BID4 BID1 BID2 BID7 BID9 BID11 BID10 BID16 goes on without cessation, indicating to a certain extent the unlimited capabilities for attackers to continuously and easily fool machine learning models. On the other hand, even state-of-the-art defense approaches such as adversarial training BID17 BID4 and defensive distillation BID12 still suffer from major limitations and can be circumvented (Section 2). Therefore, the unfortunate status quo is that attackers prevail over defenders.In this paper, from a unique angle, we propose to investigate two important research questions that concern the distinguishability of adversarial examples. Question 1: are adversarial examples distinguishable from natural examples? Question 2: are adversarial examples generated by different methods distinguishable from each other? If the answer to Question 1 will be positive, i.e., given a certain classification task such as image classification, generated adversarial examples (regardless of the objects they represent) largely belong to one class while natural examples belong to the other class, then defenders can simply discard those adversarial examples to protect the machine learning models. If the answer to Question 2 will be positive, i.e., adversarial examples generated by different methods clearly belong to different classes, defenders can better protect the machine learning models, for example, by incorporating the corresponding examples into the adversarial training process to enhance the robustness of the models. Besides such practical benefits, answering these two questions may also help researchers further identify the nature of adversarial examples.Formally, we consider a classification problem in adversarial environments as a multi-label classification problem. That is, upon seeing a new input such as an image, while the original task such as classifying the image as a certain object is important, it is also important to classify the image as a generated adversarial vs. a natural example in the first place. We formulate this multi-label classification problem in Section 3 to guide us in answering the two questions, and term the corresponding defense approach as defensive distinction, which distinguishes adversarial vs. natural examples and distinguishes adversarial examples generated by different methods to protect against the attacks.We design and perform experiments using the MNIST dataset to investigate the two research questions and evaluate the effectiveness of our defensive distinction approach. In our experiments, we consider multiple scenario-case combinations that defenders either know or do not know the neural network, source images, and methods as well as parameters used by attackers for generating adversarial examples. We obtain highly positive answers to both research questions. For example, in some typical cases, adversarial vs. natural examples can be distinguished perfectly with 100% accuracy, while adversarial examples generated by different methods can be distinguished with over 90% accuracy. Our experimental results demonstrate the strong distinguishability of adversarial examples, and demonstrate the value of the defensive distinction approach. We recommend that this unique defense approach should be seriously considered to complement other defense approaches.We make four main contributions in this paper: (1) we propose to investigate two important research questions that concern the distinguishability of adversarial examples; (2) we formulate a classification problem in adversarial environments as a multi-label classification problem to answer the two questions; (3) we propose and explore a unique defense approach termed as defensive distinction; (4) we design and perform experiments to empirically demonstrate the strong distinguishability of adversarial examples and the value of our defensive distinction approach. We proposed two important research questions that concern the distinguishability of adversarial examples, and formulated a classification problem in adversarial environments as a multi-label classification problem. We proposed a defensive distinction protection approach to answer the two questions and address the problem. We designed and performed experiments using the MNIST dataset and eight representative cases. Our experimental results demonstrate the strong distinguishability of adversarial examples, and the practical as well as research value of our approach. Our work also suggests many possibilities for the future work such as adopting high-order multi-label learning strategies to further explore the intrinsic correlations of labels as discussed in Section 3.2, investigating the distinguishability of adversarial examples for large tasks such as on ImageNet, and investigating the appropriate ways for integrating defensive distinction with other defense approaches.A APPENDIX: A POTENTIAL EXTENSION TO THE PROBLEM FORMULATION More labels could be added to include more concepts or semantic meanings in our multi-label classification formulation of the problem. For example, y i can be extended to a triplet (a i , b i , c i ) ∈ Y where Y = Y × Z × Y is a 3-ary Cartesian product, and c i ∈ Y can indicate the source example class from which the input example x i was created. In the training set, c i can simply be a i for a natural example, and is assumed to be known for an adversarial example. This more complex version of formulation has its values on further correlating to the labels of source examples, but we do not explore it in the paper.
We propose a defensive distinction protection approach and demonstrate the strong distinguishability of adversarial examples.
{"src": "scitldr-AIC_train", "paper_id": "r1glehC5tQ"}