The unprecedented success of deep neural networks in various applications have made these networks a prime target for adversarial exploitation. In this paper, we introduce a benchmark technique for detecting backdoor attacks (aka Trojan attacks) on deep convolutional neural networks (CNNs). We introduce the concept of Universal Litmus Patterns (ULPs), which enable one to reveal backdoor attacks by feeding these universal patterns to the network and analyzing the output (i.e., classifying as clean' or corrupted'). This detection is fast because it requires only a few forward passes through a CNN. We demonstrate the effectiveness of ULPs for detecting backdoor attacks on thousands of networks trained on three benchmark datasets, namely the German Traffic Sign Recognition Benchmark (GTSRB), MNIST, and CIFAR10.
Another interesting approach is Neural Cleanse @cite_29 , in which the authors propose to attack clean images by optimizing for minimal triggers that fool the pre-trained model. The rational here is that the backdoor trigger is a consistent perturbation that produces a classification result to a target class, @math , for any input image in source class @math . Therefore, the authors seek a minimal perturbation that causes the images in the source class to be classified as the target class. The optimal perturbation then could be a potential backdoor trigger. This promising approach is computationally demanding as the attacked source class might not be a priori known and such minimal perturbations should be calculated for potentially all pairs of source and target classes. In addition, a strong prior on the type of backdoor trigger is needed to be able to discriminate a potentially benign minimal perturbation from an actual backdoor trigger.
@cite_29
2741933435
Deep neural network-based classifiers are known to be vulnerable to adversarial examples that can fool them into misclassifying their input through the addition of small-magnitude perturbations. However, recent studies have demonstrated that such adversarial examples are not very effective in the physical world--they either completely fail to cause misclassification or only work in restricted cases where a relatively complex image is perturbed and printed on paper. In this paper we propose a new attack algorithm--Robust Physical Perturbations (RP2)-- that generates perturbations by taking images under different conditions into account. Our algorithm can create spatially-constrained perturbations that mimic vandalism or art to reduce the likelihood of detection by a casual observer. We show that adversarial examples generated by RP2 achieve high success rates under various conditions for real road sign recognition by using an evaluation methodology that captures physical world conditions. We physically realized and evaluated two attacks, one that causes a Stop sign to be misclassified as a Speed Limit sign in 100 of the testing conditions, and one that causes a Right Turn sign to be misclassified as either a Stop or Added Lane sign in 100 of the testing conditions.