Deep neural networks have been recently demonstrated to be vulnerable to backdoor attacks. Specifically, by altering a small set of training examples, an adversary is able to install a backdoor that can be used during inference to fully control the model’s behavior. While the attack is very powerful, it crucially relies on the adversary being able to introduce arbitrary, often clearly mislabeled, inputs to the training set and can thus be detected even by fairly rudimentary data filtering. In this paper, we introduce a new approach to executing backdoor attacks, utilizing adversarial examples and GAN-generated data. The key feature is that the resulting poisoned inputs appear to be consistent with their label and thus seem benign even upon human inspection. Over the last decade, deep learning has made unprecedented progress on a variety of notoriously difficult tasks in computer vision BID16 BID11 , speech recognition BID8 , machine translation BID28 , and game playing BID20 BID27 . Despite this remarkable performance, real-world deployment of such systems remains challenging due to concerns about security and reliability. One particular example receiving significant attention is the existence of adversarial examples -inputs with imperceptible adversarial perturbations that are misclassified with high confidence BID29 BID7 . Such adversarial perturbations can be constructed for a wide range of models, while requiring minimal model knowledge BID22 BID4 and being applicable to real-world scenarios BID26 BID17 BID1 .However , this brittleness during inference is not the only vulnerability of existing ML approaches. Another vulnerability corresponds to a different aspect of the ML pipeline: training. State-of-the-art ML models require large amounts of data to achieve good performance. Unfortunately, large datasets are expensive to generate and curate; it is hence common practice to use training examples sourced from other -often untrusted -sources. This practice is usually justified by the robustness of ML models to input and label noise BID24 ) -bad samples might only slightly degrade the model's performance. While this reasoning is valid when only benign noise is present, it breaks down when the noise is maliciously crafted. Attacks based on injecting such malicious noise to the training set are known as data poisoning attacks BID2 .A well-studied form of data poisoning aims to use the malicious samples to reduce the test accuracy of the resulting model BID31 BID21 BID19 BID3 . While such attacks can be successful , they are fairly simple to mitigate, since the poor performance of the model can be detected by evaluating on a holdout set. Another form of attack, known as targeted poisoning attacks, aims to misclassify a specific set of inputs at inference time BID14 . These attacks are harder to detect. Their impact is restricted, however, as they only affect the model's behavior on a limited, pre-selected set of inputs.Recently, BID9 proposed a backdoor attack. The purpose of this attack is to plant a backdoor in any model trained on the poisoned training set. This backdoor is activated during inference by a backdoor trigger which, whenever present in a given input, forces the model to predict a specific (likely incorrect) target label. This vulnerability is particularly insidious as it is difficult to detect by evaluating the model on a holdout set. The BID9 attack is based on randomly selecting a small portion of the training set, applying a backdoor trigger to these inputs and changing their labels to the target label. This strategy is very effective. However, it crucially relies on the BID9 Clean-label baseline GAN-based (ours) Adv. example-based (ours) Figure 1 : An example image, labeled as an airplane, poisoned using different strategies: the BID9 attack, the baseline of the same attack restricted to only clean labels, our GAN-based attack, and our adversarial examples-based attack (left to right). The original BID9 attack image is clearly mislabeled while the rest of the images appear plausible. We use the same pattern as BID9 for consistency, but our attacks use a reduced amplitude, as described in Section B.1.assumption that the poisoned inputs introduced to the training set by the adversary can be arbitraryincluding clearly mislabeled input-label pairs. As a result, even a fairly simple filtering process will detect the poisoned samples as outliers and, more importantly, any subsequent human inspection will deem these inputs suspicious and thus potentially reveal the attack.The goal of this paper is to investigate whether the usage of such clearly mislabeled (and thus suspicious) images is really necessary. That is, can such backdoor attacks be carried out when we insist that each poisoned input and its label must be consistent, even to a human? We investigate the backdoor attacks of BID9 in the presence of a simple data filtering scheme. While their attack is powerful, it crucially relies on the addition of arbitrary, mostly mislabeled, inputs into the training set and can thus be detected by filtering. Human inspection of the identified outliers will clearly flag the poisoned samples as unnatural. We argue that, for a backdoor attack to be insidious, it must not rely on inputs that appear mislabeled upon examination. To remain successful under the clean-label restriction, we propose perturbing the poisoned inputs to render them more difficult to classify. We restrict the magnitude of these changes so that the true label remains plausible.We propose two methods for increasing classification difficulty: adversarial p -bounded perturbations and GAN-based interpolation. We find that both methods introduce a backdoor more successfully than the clean-label adaptation of the BID9 attack.These findings demonstrate that backdoor attacks can be made significantly harder to detect than one might initially expect. This emphasizes the need for developing principled and effective methods for protecting ML models from such attacks.A THE ORIGINAL ATTACK OF GU We replicate the experiments of BID9 on the CIFAR-10 ( BID15 dataset. The original work considered the case where the model is trained by an adversary, since they focused on the transfer learning setting. The authors accordingly imposed essentially no constraints on the number of poisoned samples used. In contrast, we study the threat model where an attacker is only allowed to poison a limited number of samples in the dataset. We are thus interested in understanding the fraction of poisoned samples required to ensure that the resulting model indeed has an exploitable backdoor. In Figure A , we plot the attack success rate for different target labels and number of poisoned examples injected. We observe that the attack is very successful even with a small (∼ 75) number of poisoned samples. Note that the poisoning percentages here are calculated relative to the entire dataset. The x-axis thus corresponds to the same scale in terms of examples poisoned as the rest of the plots. While the attack is very effective, most image labels are clearly incorrect (Figure 1) . Despite our above focus on the plausibility of the base image, the backdoor pattern itself could also cause plausibility problems if its presence appears unnatural.To mitigate this potential suspicion, we consider a modified backdoor pattern. Instead of entirely replacing the bottom-right 3-pixel-by-3-pixel square with the pattern, we perturb the original pixel values by a backdoor pattern amplitude. In pixels that are white in the original pattern, we add this amplitude to each color channel (i.e. red, green and blue). Conversely, for black pixels, we subtract this amplitude from each channel. We then clip these values to the normal range of pixel values. (Here, the range is [0, 255] .) Note that when the backdoor pattern amplitude is 255 or greater, this attack is always equivalent to applying the original backdoor pattern. We extend our proposed adversarial example-based attack to reduced backdoor pattern amplitudes.We explore this attack with a random class (the dog class), considering backdoor pattern amplitudes of 16, 32 and 64. All (non-zero) backdoor pattern amplitudes resulted in substantial attack success rates at poisoning percentages of 6% and higher. Higher amplitudes conferred higher attack success rates. At the two lower poisoning percentages tested, the attack success rate was near zero. These results are shown in FIG8 .Image plausibility is greatly improved by reducing the backdoor pattern amplitude. Examples of an image at varying backdoor pattern amplitudes are shown in FIG9 . A more complete set of examples is available in Appendix C.3.3.We have chosen a backdoor pattern amplitude of 32 for further investigation as a balance between conspicuousness and attack success. We tested this attack on all classes, finding similar performance across the classes. These results are shown in FIG8 .
We show how to successfully perform backdoor attacks without changing training labels.
{"src": "scitldr-AIC_train", "paper_id": "HJg6e2CcK7"}