During the last years, a remarkable breakthrough has been made in AI domain
 thanks to artificial deep neural networks that achieved a great success in many
 machine learning tasks in computer vision, natural language processing, speech
 recognition, malware detection and so on. However, they are highly vulnerable
 to easily crafted adversarial examples. Many investigations have pointed out this
 fact and different approaches have been proposed to generate attacks while adding
 a limited perturbation to the original data. The most robust known method so far
 is the so called C&W attack [1]. Nonetheless, a countermeasure known as fea-
 ture squeezing coupled with ensemble defense showed that most of these attacks
 can be destroyed [6]. In this paper, we present a new method we call Centered
 Initial Attack (CIA) whose advantage is twofold : first, it insures by construc-
 tion the maximum perturbation to be smaller than a threshold fixed beforehand,
 without the clipping process that degrades the quality of attacks. Second, it is
 robust against recently introduced defenses such as feature squeezing, JPEG en-
 coding and even against a voting ensemble of defenses. While its application is
 not limited to images, we illustrate this using five of the current best classifiers
 on ImageNet dataset among which two are adversarialy retrained on purpose to
 be robust against attacks. With a fixed maximum perturbation of only 1.5% on
 any pixel, around 80% of attacks (targeted) fool the voting ensemble defense and
 nearly 100% when the perturbation is only 6%. While this shows how it is difficult
 to defend against CIA attacks, the last section of the paper gives some guidelines
 to limit their impact. Since the skyrocketing of data volumes and parallel computation capacities with GPUs during the last years, deep neural networks (DNN) have become the most effective approaches in solving many machine learning problems in several domains like computer vision, speech recognition, games playing etc. They are even intended to be used in critical systems like autonomous vehicle BID17 , BID18 . However, DNN as they are currently built and trained using gradient based methods, are very vulnerable to attacks a.k.a. adversarial examples BID1 . These examples aim to fool a classifier to make it predict the class of an input as another one, different from the real class, after bringing only a very limited perturbation to this input. This can obviously be very dangerous when it comes to systems where human life is in stake like in self driven vehicles. Companies IT networks and plants are also vulnerable if DNN based intrusion detection systems were to be deployed BID20 .Many approaches have been proposed to craft adversarial examples since the publication by Szegedy et al. of the first paper pointing out DNN vulnerability issue BID4 . In their work, they generated adversarial examples using box-constrained L-BFGS. Later in BID1 , a fast gradient sign method (FGSM) that uses gradients of a loss function to determine in which direction the pixels intensity should be changed is presented. It is designed to be fast not optimize the loss function. Kurakin et al. introduced in BID13 a straightforward simple improvement of this method where instead of taking a single step of size in the direction of the gradient-sign, multiple smaller steps are taken, and the result is clipped in the end. Papernot et al. introduced in BID3 an attack, optimized under L0 distance, known as the Jacobian-based Saliency Map Attack (JSMA). Another simple attack known as Deepfool is provided in [34] . It is an untargeted attack technique optimized for the L2 distance metric. It is efficient and produces closer adversarial examples than the L-BFGS approach discussed earlier. Evolutionary algorithms are also used by authors in BID14 to find adversarial example while maintaining the attack close to the initial data. More recently, Carlini and Wagner introduced in BID0 the most robust attack known to date as pointed out in BID5 . They consider different optimization functions and several metrics for the maximum perturbation. Their L2-attack defeated the most powerful defense known as distillation BID7 . However, authors in BID6 showed that feature squeezing managed to destroy most of the C&W attacks. Many other defenses have been published, like adversarial training BID3 , gradient masking BID9 , defenses based on uncertainty using dropout BID10 as done with Bayesian networks, based on statistics BID11 , BID12 , or principal components BID22 , BID23 . Later, while we were carrying out our investigation, paper BID16 showed that not less than ten defense approaches, among which are the previously enumerated defenses, can be defeated by C&W attacks. It also pointed out that feature squeezing also can be defeated but no thorough investigation actually was presented. Another possible defense but not investigated is based on JPEG encoding when dealing with images. It has never been explicitly attacked even after it is shown in BID13 that most attacks are countered by this defense. Also, to our knowledge, no investigation has been conducted when dealing with ensemble defenses. Actually, attacks transferability between models that is well investigated and demonstrated in BID19 in the presence of an oracle (requesting defense to get labels to train a substitute model) is not guaranteed at all when it is absent. Finally, when the maximum perturbation added to the original data is strictly limited, clipping is needed at the end of training (adversarial crafting) even if C&W attacks are used. The quality of crafted attacks is therefore degraded as the brought perturbation during the training is brutally clipped. We tackle all these points in our work while introducing a new attack we call Centered Initial Attack (CIA). This approach considers the perturbation limits by construction and consequently no alteration is done on the CIA resulting examples.To make it clearer for the reader, an example is given below to illustrate the clipping issue. FIG0 shows a comparison between CIA and C&W L2 attack before and after clipping on an example, a guitar targeted as a potpie with max perturbation equal to 4.0 (around 1.5%). The same number of iterations FORMULA4 is considered for both methods. As can be seen on FIG0 , CIA generates the best attack with 96% confidence. C&W is almost as good with a score of 95% but it is degraded to 88% after applying the clipping to respect the imposed max perturbation. Avoiding this degradation due to clipping is the core motivation of our investigation.The remaining of this paper is organized as follows. Section I presents some mathematical formulations and the principle of CIA strategy. Then Section II investigates the application of CIA against ensemble defense, feature squeezing and JPEG encoding defenses. Then Section III provides some guidelines to limit the impact of CIA attacks. Finally, we give some possible future investigations in the conclusion . In this paper we presented a new strategy called CIA for crafting adversarial examples while insuring the maximum perturbation added to the original data to be smaller than a fixed threshold. We demonstrated also its robustness against some defenses, feature squeezing, ensemble defenses and even JPEG encoding. For future work, it would be interesting to investigate the transferability of CIA attacks to the physical world as it is shown in BID13 that only a very limited amount of FGDM attacks, around 20%, survive this transfer. Another interesting perspective is to consider partial crafting attacks while selecting regions taking into account the content of the data. With regard to images for instance, this would be interesting to hide attacks with big but imperceptible perturbations.
 In this paper, a new method we call Centered Initial Attack (CIA) is provided. It insures by construction the maximum perturbation to be smaller than a threshold fixed beforehand, without the clipping process.
{"src": "scitldr-AIC_train", "paper_id": "rJqfKPJ0Z"}