Neural networks with low-precision weights and activations offer compelling
 efficiency advantages over their full-precision equivalents. The two most
 frequently discussed benefits of quantization are reduced memory consumption,
 and a faster forward pass when implemented with efficient bitwise
 operations. We propose a third benefit of very low-precision neural networks:
 improved robustness against some adversarial attacks, and in the worst case,
 performance that is on par with full-precision models. We focus on the very
 low-precision case where weights and activations are both quantized to $\pm$1,
 and note that stochastically quantizing weights in just one layer can sharply
 reduce the impact of iterative attacks. We observe that non-scaled binary neural
 networks exhibit a similar effect to the original \emph{defensive distillation}
 procedure that led to \emph{gradient masking}, and a false notion of security.
 We address this by conducting both black-box and white-box experiments with
 binary models that do not artificially mask gradients. The ability to fool machine learning models by making small changes to their input severely limits their potential for safe use in many real-world scenarios. Example vulnerabilities include a seemingly innocuous audio broadcast that is interpreted by a speech recognition model in a smartphone, with the intent to trigger an e-transfer, as well as pictures or identity documents that are automatically tagged as someone other than the real individual.The two most common threat models when evaluating the security of a system are the black-box and white-box assumptions, which represent varying degrees of information that an adversary may possess. In a black-box threat model, an adversary has similar abilities to a normal user that interacts with a system by providing inputs and observing the corresponding outputs. Under this threat model, an adversary generally does not know details of the model architecture or dataset used to train the model. Of course, an adversary is free to assume that a convolutional architecture was likely used if the input domain is images, or a recurrent model for speech or text.In a white-box threat model, an adversary has complete access to the model architecture and parameters. In the case of neural networks, white-box attacks frequently rely on gradient information to craft especially strong adversarial examples, where strong means that the example is very close to the original input as defined by some distance norm (e.g. L 0 -number of features modified, L 2 -mean squared distance), yet is very likely to cause the model to yield the incorrect output. For both threat types, targeted attacks where a model is made to fail in a specific way (e.g. causing a handwritten '7' look like a '3') represents a stronger attack than simple misclassification.The problem with deploying machine learning systems that are secured in a traditional sense, is that adversarial examples have been shown to generalize well between models with different source and target architectures BID19 BID16 Tramèr et al., 2017) . This means that a secured model can be compromised in an approximately white-box setting by training and attacking a substitute model that approximates the decision boundary of the model under attack BID16 . Thus, to make strong conclusions about the robustness of a machine learning model to adversarial attacks, both threat models should be considered.Tangent to research on defences against adversarial attacks, significant progress has been made towards training very low-precision deep neural networks to accuracy levels that are competitive with full-precision models BID4 BID22 BID20 . The current motivation for extreme quantization is the ability to deploy these models under hardware resource constraints, acceleration, or reduced power consumption. Ideally, 32× compression is possible by using 1-bit to represent single-precision floating point parameters. By similarly quantizing activations, we can reduce run-time memory consumption as well. These savings enable large scale deployment of neural networks on the billions of existing embedded devices. Very low-precision models were designed with deployment in mind, and may be responsible for making critical decisions in embedded systems, all subject to reverse engineering and a diverse set of real world attacks. With much at stake in applications like autonomous navigation, robotics, and network infrastructure, understanding how very low-precision neural networks behave in adversarial settings is essential. To that end, we make the following contributions:• To the best of our knowledge, we are the first to formally evaluate and interpret the robustness of binary neural networks (BNNs) to adversarial attacks on the MNIST BID10 Cortes, 1998) and CIFAR-10 (Krizhevsky, 2009 ) datasets.• We compare and contrast the properties of low-precision neural networks that confer adversarial robustness to previously proposed defense strategies. We then combine these properties to propose an optimal defense strategy.• We attempt to generalize and make recommendations regarding the suitability of lowprecision neural networks against various classes of attacks (e.g. single step vs. iterative). We suspect that plain BNNs implement two different kinds of gradient masking. We discovered the first by tracking the L1 norm of the hidden layer activations and unscaled logits. BNNs operate with larger range and variance than 'normal' networks, which can be explained by virtue of convolving inputs with greater magnitude (±1) compared with the typically small values taken by weights and activations. For our 64 kernel CNN, the logits were about 4× larger than the scaled or full-precision networks. This is analogous to the more complex defensive distillation procedure in which the model to be secured is trained with soft-labels generated by a teacher model. When training the teacher, a softmax temperature, T 1 is used. The distilled model is trained on the labels assigned by the teacher and using the same T . At test time, the model is deployed with T = 1, which causes the logits to explode with respect to their learned values. The logits saturate the softmax function and cause gradients to vanish, leading FGSM and JSMA to fail at a higher rate. However, this defense is defeated with a close enough guess for T , or via a black box attack BID3 .The second type of gradient masking is less easily overcome, and has to do with gradients being inherently discontinuous and non-smooth, as seen in FIG1 of Appendix B. We believe that this effect is what gives scaled BNNs an advantage over full-precision with respect to targeted attacks. Even more importantly, through a regularization effect, the decision boundary for the MLP with binary units FIG1 ) better represents the actual function to be learned, and is less susceptible to adversarial examples. But why does gradient masking have a disproportionate effect when attacking compared with training on clean inputs? Models 'A' and 'B' were trained to within 1.2% test accuracy, while 'B' had improvements of 9.0% and 29.5% on JSMA and CWL2 attacks respectively, corresponding to 8× and 25× difference in accuracy, respectively, for adversarial vs. clean inputs. For JSMA , the performance gap can be attributed to the sub-optimality of the attack as it uses logits rather than softmax probabilities. Furthermore , to achieve its L0 goal, pairs of individual pixels are manipulated which is noisy process in a binarized model. The success of model 'S' with stochastically quantized weights in its third convolutional layer against iterative attacks is more easily explained. Adversarial examples are not random noise, and do not occur in random directions. In fact, neural networks are extremely robust to large amounts of benign noise. An iterative attack that attempts to fool our stochastically quantized model faces a unique model at every step, with unique gradients. Thus, the direction that minimizes the probability of the true class in the first iteration is unlikely to be the same in the second. An iterative attack making n steps is essentially attacking an ensemble of n models. By making a series of small random steps, the adversary is sent on the equivalent of a wild goose chase and has a difficult time making progress in any particularly relevant direction to cause an adversarial example. We have shown that for binarized neural networks, difficulty in training leads to difficulty when attacking. Although we did not observe a substantial improvement in robustness to single step attacks through binarization, by introducing stochasticity we have reduced the impact of the strongest attacks. Stochastic quantization is clearly far more computationally and memory efficient than a traditional ensemble of neural networks, and could be run entirely on a micro-controller with a pseudo random number generator. Our adversarial accuracy on MNIST against the best white-box attack (CWL2) is 71±2% (S64+) compared with the best full-precision model 1.8±0.9% (A256+). Blackbox results were competitive between binary and full-precision on MNIST, and binary models were slightly more robust for CIFAR-10, which we attribute to their improved regularization. Beyond their favourable speed and resource usage, we have demonstrated another benefit of deploying binary neural networks in industrial settings. Future work will consider other types of low-precision models as well as other adversarial attack methods. Table 6 : Error on clean MNIST test set for models with varying capacity and precision. A is fullprecision, B is binary, and C is binary with a learned scalar applied to the ReLU in hidden layers. All models were trained with Adam for 15 epochs with a batch size of 128 and a learning rate of 1e-3.
We conduct adversarial attacks against binarized neural networks and show that we reduce the impact of the strongest attacks, while maintaining comparable accuracy in a black-box setting
{"src": "scitldr-AIC_train", "paper_id": "HkTEFfZRb"}