Recently, researchers have discovered that the state-of-the-art object classifiers can be fooled easily by small perturbations in the input unnoticeable to human eyes. It is known that an attacker can generate strong adversarial examples if she knows the classifier parameters. Conversely, a defender can robustify the classifier by retraining if she has the adversarial examples. The cat-and-mouse game nature of attacks and defenses raises the question of the presence of equilibria in the dynamics. In this paper, we present a neural-network based attack class to approximate a larger but intractable class of attacks, and formulate the attacker-defender interaction as a zero-sum leader-follower game. We present sensitivity-penalized optimization algorithms to find minimax solutions, which are the best worst-case defenses against whitebox attacks. Advantages of the learning-based attacks and defenses compared to gradient-based attacks and defenses are demonstrated with MNIST and CIFAR-10. Recently, researchers have made an unsettling discovery that the state-of-the-art object classifiers can be fooled easily by small perturbations in the input unnoticeable to human eyes BID24 BID7. Following studies tried to explain the cause of the seeming failure of deep learning toward such adversarial examples. The vulnerability was ascribed to linearity BID24, low flexibility BID4, or the flatness/curvedness of decision boundaries BID20, but a more complete picture is still under research. This is troublesome since such a vulnerability can be exploited in critical situations such as an autonomous car misreading traffic signs or a facial recognition system granting access to an impersonator without being noticed. Several methods of generating adversarial examples were proposed BID7 BID19 BID2, most of which use the knowledge of the classifier to craft examples. In response, a few defense methods were proposed: retraining target classifiers with adversarial examples called adversarial training BID24 BID7; suppressing gradient by retraining with soft labels called defensive distillation BID22; hardening target classifiers by training with an ensemble of adversarial examples BID25.In this paper we focus on whitebox attacks, that is, the model and the parameters of the classifier are known to the attacker. This requires a more robust classifier or defense method than simply relying on the secrecy of the parameters as defense. When the classifier parameters are known to an attacker, existing attack methods are very successful at fooling the classifiers. Conversely, when the attack is known to the classifier, e.g., in the form of adversarial examples, one can weaken the attack by retraining the classifier with adversarial examples, called adversarial training. However, if we repeat adversarial sample generation and adversarial training back-to-back, it is observed that the current adversarially-trained classifier is no longer robust to previous attacks (see Sec. 3.1.) To find the classifier robust against the class of gradient-based attacks, we first propose a sensitivitypenalized optimization procedure. Experiments show that the classifier from the procedure is more robust than adversarially-trained classifiers against previous attacks, but it still remains vulnerable to some degrees. This raises the main question of the paper: Can a classifier be robust to all types of attacks? The answer seems to be negative in light of the strong adversarial examples that can be crafted by direct optimization procedures from or BID2. Note that the class of optimization-based attack is very large, as there is no restriction on the adversarial patterns that can be generated except for certain bounds such as l p -norm bounds. The vastness of the optimization-based attack class is a hindrance to the study of the problem, as the defender cannot learn efficiently about the attack class from a finite number of samples. To study the problem analytically, we use a class of learning-based attack that can be generated by a class of neural networks. This class of attack can be considered an approximation of the class of optimization -based attacks, in that the search space of optimal perturbation is restricted to the parameter space of a neural network architecture, e.g., all perturbations that can be generated by fully-connected 3-layer ReLU networks. Similar to what we propose, others have recently considered training neural networks to generate adversarial examples BID21 BID0. While the proposed learning-based attack is weaker than the optimization-based attack, it can generate adversarial examples in test time with only single feedforward passes, which makes real-time attacks possible. We also show that the class of neural-network based attacks is quite different from the the class of gradient-based attacks (see Sec. 4.1.) Using the learning-based attack class, we introduce a continuous game formulation for analyzing the dynamics of attack-defense. The game is played by an attacker and a defender/classifier 1, where the attacker tries to maximize the risk of the classification task by perturbing input samples under certain constraints such as l p -norm bounds, and the defender/classifier tries to adjust its parameters to minimize the same risk given the perturbed inputs. It is important to note that for adversarial attack problems, the performance of an attack or a defense cannot be measured in isolation, but only in pairs of (attack, defense). This is because the effectiveness of an attack/defense depends on the defense/attack it is against. As a two-player game, there may not be a dominant defense that is no less robust than all other defenses against all attacks. However, there is a natural notion of the best defense or attack in the worst case. Suppose one player moves first by choosing her parameters and the other player responds with the knowledge of the first player's move. This is an example of a leader-follower game BID1 for which there are two well-known states, the minimax and the maximin solutions if it is a constant-sum game. To find those solutions empirically, we propose a new continuous optimization method using the sensitivity penalization term. We show that the minimax solution from the proposed method is indeed different from the solution from the conventional alternating descent/ascent and is also more robust. We also show that the strength/weakness of the minimax-trained classifier is different from that of adversarially-trained classifiers for gradient-based attacks. The contributions of this paper are summarized as follows.• We provide a continuous game model to analyze adversarial example attacks and defenses, using the neural network-based attack class as a feasible approximation to a larger but intractable class of optimization-based attacks.• We demonstrate the difficulty of defending against multiple attack types and present the minimax defense as the best worst-case defense methods.• We propose a sensitivity-penalized optimization method (Alg. 1) to numerically find continuous minimax solutions, which is better than alternating descent/ascent. The proposed optimization method can also be used for other minimax problems beyond the adversarial example problem. The proposed methods are demonstrated with the MNIST and the CIFAR-10 datasets. For readability, details about experimental settings and the with CIFAR-10 are presented in the appendix. Making a classifier robust to test-time adversarial attacks has been studied for linear (kernel) hyperplanes BID13, naive Bayes BID3 and SVM BID5, which also showed the game-theoretic nature of the robust classification problems. Since the recent discovery of adversarial examples for deep neural networks, several methods of generating adversarial samples were proposed BID24 BID7 BID19 BID2 as well as several methods of defense BID24 BID7 BID22 BID25. These papers considered static scenarios, where the attack/defense is constructed against a fixed opponent. A few researchers have also proposed using a detector to detect and reject adversarial examples BID16 BID14 BID18. While we do not use detectors in this work, the minimax approach we proposed in the paper can be applied to train the detectors. The idea of using neural networks to generate adversarial samples has appeared concurrently BID0 BID21. Similar to our paper, the two papers demonstrates that it is possible to generate strong adversarial samples by a learning approach. BID0 explored different architectures for the "adversarial transformation networks" against several different classifiers. BID21 proposed "attack learning neural networks" to map clean samples to a region in the feature space where misclassification occurs and "defense learning neural networks" to map them back to the safe region. Instead of prepending the defense layers before the fixed classifier BID21, we retrain the whole classifier as a defense method. However, the key difference of our work to the two papers is that we consider the dynamics of a learning-based defense stacked with a learning-based attack, and the numerical computation of the optimal defense/attack by continuous optimization. The alternating gradient-descent method for finding an equilibrium of a game has gained renewed interest since the introduction of Generative Adversarial Networks (GAN) BID6. However, the instability of the alternating gradient-descent method has been known, and the "unrolling" method BID17 was proposed to speed up the GAN training. The optimization algorithm proposed in the paper has a similarity with the unrolling method, but it is simpler (corresponding to a single-step unrolling) and involves a gradient-norm regularization which can be interpreted intuitively as sensitivity penalization BID8 BID15. Lastly, the framework of minimax risks was also studied in BID9 for the purpose of privacy preservation. We propose a different algorithm in this paper, but we also show that the attack on classification and the attack on privacy are the two sides of the same optimization problem with the opposite goals.3 CAT-AND-MOUSE GAME A classifier whose parameters are known to an attacker is easy to attack. Conversely, an attacker whose sample-generating method is known to a classifier is easy to defend from. In this section, we demonstrate the cat-and-mouse nature of the interaction, using adversarial training (Adv Train) as defense and the fast gradient sign method (FGSM) BID7 and the iterative version (IFGSM) BID11 as attacks. We then show that the equilibrium, if it exists, can be found more efficiently by directly solving a sensitivity-penalized optimization problem. Suppose g is a classifier g: X → Y and l(g(x), y) is a loss function. The FGSM attack generates a perturbed example z(x) given the clean sample x as follows: DISPLAYFORM0 The clean input images we use here are l ∞ -normalized, that is, all pixel values are in the range [−1, 1]. It was argued that the use of true label y in "label leaking" BID12 ), but we use will true labels in the paper for simplicity. For another attack example, the IFGSM attack iteratively refines an adversarial example by the following update DISPLAYFORM1 where the clipping used in this paper is clip x,η (x) min{1, x + η, max{−1, x − η, x}}.Existing attack methods such as FGSM and IFGSM are very effective at fooling the classifier. Table 1 shows that the two methods are able to perfectly fool a convolutional neural network trained with clean images from MNIST. (Details of the classifier architecture and the settings are in the appendix.)On the other hand, these attacks, if known to the classifier, can be weakened by retraining the classifier with the original dataset augmented by adversarial examples with ground-truth labels, known as adversarial training. In this paper we use the 1:1 mixture of the clean and the adversarial samples for adversarial training. Table 2 shows the of adversarial training for different attacks. Defense\Attack No attack FGSM IFGSM η=0.3 η=0.4 η=0.5 η=0.6 η=0.3 η=0.4 η=0.5 η=0.6 No defense 0.006 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Table 1: Test error rates of FGSM and IFGSM attacks on an undefended convolutional neural network for MNIST. These attacks can cause perfect misclassification for the given range of η. The test error rates for adversarial test examples after training become below 1% indicating nearperfect avoidance. This is in stark contrast with the perfect misclassification of the undefended classifier in Table 1.Defense\Attack No attack FGSM IFGSM η=0.3 η=0.4 η=0.5 η=0.6 η=0.3 η=0.4 η=0.5 η=0.6 Adv train n/a 0.004 0.003 0.003 0.005 0.003 0.003 0.004 0.010 Table 2: Error rates of FGSM and IFGSM attacks on adversarially-trained classifiers for MNIST. This defense can avert the attacks and achieve the error rates of the no-attack case. A question arises as to what would happen if the procedure of 1) adversarial sample generation using the current classifier, and 2) retraining classifier using the current adversarial examples is repeated for many rounds. The answer to this cat-and-mouse game is easy to experiment although time-consuming. Let's denote the attack on the original classifier as FGSM1, and the corresponding retrained classifier as Adv FGSM1. Repeating the procedure above generates the sequence of models FGSM1 → Adv FGSM1 → FGSM2 → Adv FGSM2, etc. FIG0 shows one such trial with 80 + 80 rounds of the procedure. Initially, the attacker achieves near-perfect attacks (i.e., error rate 1), and the defender achieves near-perfect defense (i.e., error rate 0). As the iteration increases, the attacker becomes weaker with error rate 0.5, but the defense is still very successful, and the rate seems to oscillate persistently. While we can run more iterations to see if it converges, this is not a very principled nor efficient approach to find an equilibrium, if it exists. We can perform the cat-and-mouse simulation more efficiently by an optimization approach. Instead of training the classifier fully with adversarial examples and then regenerating adversarial examples, suppose we only update the classifier with a single gradient-descent step then regenerate adversarial examples. To emphasize the parameters u of the classifier/defender g(x; u), let's rewrite the empirical risk of classifying the perturbed data as DISPLAYFORM0 where z(x) denote an FGSM-like attack based on the loss gradient DISPLAYFORM1 and DISPLAYFORM2 ) is the sequence of perturbed examples. In expectation of the attack, the defender should choose u to minimize f (u, Z(u)) where the dependence of the attack on the classifier u is expressed explicitly. If we minimize f using gradient descent DISPLAYFORM3 then from the chain rule, the total derivative DISPLAYFORM4 from FORMULA2 and.Interestingly, this total derivative at the current state coincides with the gradient ∇ u of the following cost DISPLAYFORM5 where γ = ηN. There are two implications. Interpretation-wise, this cost function is the sum of the original risk f and the'sensitivity' term ∂f /∂Z 2 which penalizes abrupt changes of the risk w.r.t. the input. Therefore, u is chosen at each iteration to not only decrease the risk but also to make the classifier insensitive to input perturbation so that the attacker cannot take advantage of large gradients. The idea of minimizing the sensitivity to input is a familiar approach in robustifying classifiers BID8 BID15. Secondly, the new formulation can be implemented easily. The gradient descent update using the seemingly complicated gradient can be replaced by the gradient descent update of. The capability of automatic differentiation BID23 in modern machine learning libraries can be used to compute the gradient of FORMULA7 efficiently. Using this direct approach, we can find the defense parameters u which will be robust to gradient-based attacks. Fig. 2 shows the decrease of test error during training using the this gradient descent approach for MNIST. It only takes a very small fraction of time to reach the final states of the There is also an important difference between the solution of the cat-and-mouse game and the minimizer of. Table 3 shows that the adversarially trained classifier (Adv FGSM1) is robust to both clean data and FGSM1 attack, but is susceptible to FGSM2 attack, displaying the cat-and-mouse nature. The same holds for Adv FGSM2, Adv FGSM3, etc. After 80 rounds of the cat-and-mouse procedure, the classifier Adv FGSM80 becomes robust to FGSM80 as well as moderately robust to other attacks including FGSM81 (=FGSM-curr). However, the classifier Sens FGSM from direct minimization of FORMULA7 is even more robust toward FGSM-curr than Adv FGSM80 and is overall the best. To see the advantage of the sensitivity term in FORMULA7, we also performed the minimization of FORMULA7 without the sensitivity term under the same conditions as Sens FGSM. This optimization method is similar to the method proposed in, referred to as Learning with Adversaries (LWA FGSM). In the table, one can see that Sens FGSM is also better than LWA FGSM overall, although the difference is small. Note that Sens FGSM is better than other adversarially-trained classifiers, it too is still vulnerable to attacks such as FGSM80. This vulnerability raises the question if it is possible to make a classifier robust to any type of attacks, or more practically, robust to at least a large class of attacks. We discuss this issue in the next section. Table 3: Error rates of different attacks on various adversarially-trained classifiers for MNIST. FGSM-curr means the FGSM attack on the specific classifier on the left. Adv FGSM is the classifier adversarially trained with FGSM attacks. Sens FGSM is the of minimizing FORMULA7 by gradient descent. LWA FGSM is the of minimizing FORMULA7 without the gradient-norm term. In this section, we consider the class of optimization-based attack and the class of neural-network based attacks as an approximation of the former. Using the neural-network based attack class, we formulate the attacker-defender dynamics as a game and discuss two types of equilibria -the minimax and the maximin solutions. We present algorithms that generalize the approach presented in the previous section. An attacker z(x): X → X can be more general than a specific class of attacks such as FGSM. Again, let g: X → Y is a classifier parameterized by u and l(g(x; u), y) is a loss function. If time complexity is not an issue, the following optimization-based attack max DISPLAYFORM0 which is also related to the CW attack BID2, can generate strong adversarial examples, where adversarial patterns Z = (z 1, ..., z N) are unrestricted except for the bounds such as z i − x i p ≤ η. The corresponding class of adversarial patterns Z is very large, which in strong but non-generalizable adversarial examples. Non-generalizable means the perturbation z(x) has to be recomputed for every new test sample x. While the class of optimization-based attacks is powerful, its large size makes it difficult to analytically study the optimal defense methods. To make the problem learnable, we restrict the class of patterns Z to that which can be generated by a flexible but manageable class of perturbation {z(·; v) | ∀v ∈ V }, e.g., an autoencoder of a fixed architecture where the parameter v is the network weights. This class is a clearly an approximation to the class of full optimization-based attacks, but is generalizable, i.e., no time-consuming optimization is required in the test phase but only single feedforward passes. The attack network (AttNet), as we call it, can be of any class of appropriate neural networks. Here we use a three-layer fully-connected network with 300 hiddens units per layer in this paper. Different from BID21 or BID0, we feed the label y into the input of the network along with the features x. This is analogous to using the true label y in the original FGSM. While this label input is optional but it can make the training of the attacker network easier. As with other attacks, we impose the l ∞ -norm constraint on z, i.e., z(x) − x ∞ ≤ η. Suppose now f (u, v) is the empirical risk of a classifier-attacker pair where the input x is first transformed by attack network z(x; v) and then fed to the classifier g(z(x; v); u). The attack network can be trained by gradient descent as well. Given a classifier u, we can use gradient descent DISPLAYFORM1 to find an optimal attacker v that maximizes the risk f assuming the classifier u is fixed. Table 4 compares the error rates of the FGSM attacks and the attack network (AttNet). The table shows that AttNet is better than or comparable to FGSM in all cases. In particular, we already observed that the FGSM attack is no more effective against the classifier hardened against gradient-based attacks (Adv FGSM80 or Sens FGSM), but the AttNet can incur significant error (>∼ 0.9) for those hardened defenders. This indicates that the class of learning-based attacks is indeed different from the class of gradient-based attacks. Table 4: Error rates of FGSM vs learning-based attack network (AttNet) on various adversariallytrained classifiers for MNIST. FGSM-curr/AttNet-curr means they are computed/trained for the specific classifier on the leftmost column. Note that FGSM fails to attack hardened networks (Adv FGSM80 and Sens FGSM), whereas AttNet can still attack them successfully. Finally, we consider the dynamics of the pair of classifier-attacker when each player can change its parameters. Given the current classifier u, an optimal whitebox attacker parameter v is the maximizer of the risk DISPLAYFORM0 Consequently, the defender should choose the classifier parameters u such that the maximum risk is minimized u * arg min DISPLAYFORM1 This solution to the continuous minimax problem has a natural interpretation as the best worst-case solution. Assuming the attacker is optimal, i.e., it chooses the best attack from given u, no other defense can achieve a lower risk than the minimax defense u * in. The minimax defense is also a conservative defense. If the attacker is not optimal, and/or if the attack does not know the defense u exactly (as in blackbox attacks), the actual risk can be lower than what the minimax solution f (u *, v * (u *)) predicts. Before proceeding further, we point out that the claims above apply to the global minimizer u * and the maximizer function v * (·), but in practice we can only find local solutions for complex risk functions of deep classifiers and attackers. To solve FORMULA0, we analyze the problem similarly to- FORMULA7 from the previous section. At each iteration, the defender should choose u in expectation of the attack and minimize f (u, v * (u)). We use gradient descent DISPLAYFORM2 where the total derivative DISPLAYFORM3 Since the exact maximizer v * (u) is difficult to find, we only update v incrementally by one (or more) steps of gradient-ascent update DISPLAYFORM4 The ing formulation is closely related to the unrolled optimization BID17 proposed for training GANs, although the latter has a very different cost function f. Using the single update, the total derivative is DISPLAYFORM5 Similar to hardening a classifier against gradient-based attacks by minimizing FORMULA7 at each iteration, the gradient update of u for f (u, v) can be done using the gradient of the following sensitivitypenalized function DISPLAYFORM6 In other words, u is chosen not only to minimize the risk but also to prevent the attacker from exploiting the sensitivity of f to v. The algorithm is summarized in Alg. 1. Note that this algorithm is actually independent of the adversarial example problem, and can be used for other minimax problems as well. In analogy with the minimax problem, we can also consider the maximin solution defined by DISPLAYFORM0 where DISPLAYFORM1 is the minimizer function. Here we are abusing the notations for the minimax solution u *, the maximin solution v *, the minimizer u * (·), and the maximizer v * (·). Similar to the minimax solution, the maximin solution has an intuitive meaning -it is the best worst-case solution for the attacker. Assuming the defender is optimal, i.e., it chooses the best defense from that minimizes the risk f (u, v) given the attack v, no other attack can inflict a higher risk than the maximin attack v *. It is also a conservative attack. If the defender is not optimal, and/or if the defender does not know the attack v exactly, the actual risk can be higher than what the solution f (u * (v *), v * ) predicts. Note that the maximin scenario where the defender knows the attack method is not very realistic but is the opposite of the minimax scenario and provides the lower bound. To summarize, minimax and maximin defenses and attacks have the following inherent properties. Lemma 1. Let u *, v * (u), v *, u * (v) be the solutions of FORMULA0, FORMULA0, FORMULA0, FORMULA0. DISPLAYFORM2 For any given defense u, the max attack v * (u) is the most effective attack. DISPLAYFORM3 Against the optimal attack v * (u), the minimax defense u * is the most effective defense. DISPLAYFORM4 For any given attack v, the min defense u * (v) is the most effective defense. DISPLAYFORM5 Against the optimal defense u * (v), the maximin attack v * is the most effective attack. DISPLAYFORM6 The risk of the best worst-case attack is lower than that of the best worst-case defense. These properties follow directly from the definitions. The lemma helps us to better understand the dependence of defense and attack, and gives us the range of the possible risk values which can be measured empirically. To find maximin solutions, we use the same algorithm (Alg. 1) except that the variables u and v are switched and the sign of f is flipped before the algorithm is called. In addition to minimax and maximin optimization, we also consider as a reference algorithm the alternating descent/ascent method used in GAN training BID6 DISPLAYFORM0 Note that alternating descent/ascent finds local saddle points which are not necessarily minimax or maximin solutions, and therefore its solution will in general be different from the solution from Alg. 1. The difference of the solutions from three optimizations -Minimax, Maximin, and Alternating descent/ascent (Alt) -applied to a common problem, is demonstrated in FIG2. The figure shows the test error over the course of optimization starting from random initializations. One can see that Minimax (top blue curves) and Alt (middle green curves) converge to different values suggesting the learned classifiers will also be different. Table 5 compares the robustness of the classifiers trained by Minimax and Alt against the AttNet attack (1st/2nd rows and 2nd column for each η.) Minimax defense is more robust than Alt defense at η = 0.3 (0.020 vs 0.104) and at η = 0.4 (0.552 vs 0.873). For larger η's, both are unusably vulnerable. Different performance of the two classifiers implies that the minimax solution found by Alg. 1 is different from the local saddle point found by alternating descent/ascent. In addition, against FGSM attacks, Minimax is moderately robust (0.218 -0.342) despite that the classifiers are not specifically trained against gradient-based attacks. In contrast, Sens FGSM is very vulnerable (0.902 -1.000) against AttNet which we have already observed. This suggests that the class of AttNet attacks and the class of gradient-based attacks are indeed different, and the former class is larger than the latter. Table 5: Error rates of Minimax-, Alt-, and adversarially-trained (Sens FGSM) classifiers for MNIST. Minimax is overall better than Alt against AttNet-curr, and is also moderately robust against the out-of-class attack (FGSM-curr).Lastly, the adversarial examples generated by various attacks in the paper have diverse patterns and are shown in FIG4 of the appendix. We discuss some limitations of the framework and also propose an extension. Ideally, a defender should find a robust classifier against the worst attack from a very large class of attacks such as optimization-based attacks. However, it is difficult to train classifiers against attacks from a large class. On the other hand, if the class is too small, then the worst attack from that class is not representative of all possible worst attacks, and therefore the minimax defense found will not be robust to out-of-class attacks. The trade-off seems inevitable. It is, however, possible to build a defense against multiple specific types of attacks. Suppose z 1 (u),..., z m (u) are m different types of attacks, e.g., z 1 =FGSM, z 2 =IFGSM, etc. The minimax defense for the combined attack is the solution to the mixed continuous-discrete problem DISPLAYFORM0 Additionally, suppose z m+1 (u, v),..., z m+n (u, v) are n different types of learning-based attacks, e.g., z m+1 =2-layer dense net, z m+2 =5-layer convolutional nets, etc. The minimax defense against the mixture of multiple fixed-type and learning-based attacks can be found by solving DISPLAYFORM1 Due to the huge computational demand to solve, we leave it as a future work. Lastly, we discuss a bigger picture of the game between adversarial players. The minimax optimization arises in the leader-follower game BID1 with the constant sum constraint. The leader-follower setting makes sense because the defense (=classifier parameters) is often public knowledge and the attacker exploits the knowledge. Interestingly, the problem of the attack on privacy BID9 has a very similar formulation as the adversarial attack problem, different only in that the classifier is an attacker and the data perturbator is a defender. In the problem of privacy preservation against inference, the defender is a data transformer z(x) (parameterized by u) which perturbs the raw data, and the attacker is a classifier (parameterized by v) who tries to extract sensitive information such as identity from the perturbed data such as online activity of a person. The transformer is the leader, such as when the privacy mechanism is public knowledge, and the classifier is the follower as it attacks the given perturbed data. The risk for the defender is therefore the accuracy of the inference of sensitive information measured by −E[l(z(x; u), y; v)]. Solving the minimax risk problem (min u max v −E[l(z(x; u), y; v)]) gives us the best worst-case defense when the classifier/attacker knows the transformer/defender parameters, which therefore gives us a robust data transformer to preserve the privacy against the best inference attack (among the given class of attacks.) On the other hand, solving the maximin risk problem (max v min u −E[l(z(x; u), y; v)]) gives us the best worst-case classifier/attacker when its parameters are known to the transformer. As one can see, the problems of adversarial attack and privacy attack are two sides of the same coin which can be addressed by similar frameworks and optimization algorithms. In this paper, we present a continuous game formulation of adversarial attacks and defenses using a learning-based attack class implemented by neural networks. We show that this class of attacks is quite different from the gradient-based attacks. While a classifier robust to all types of attack may yet be an elusive goal, the minimax defense against the neural network-based attack class is well-defined and practically achievable. We show that the proposed optimization method can find minimax defenses which are more robust than adversarially-trained classifiers and the classifiers from simple alternating descent/ascent. We demonstrate these with MNIST and CIFAR-10. The architecture of the MNIST classifier is similar to the Tensorflow model 2, and is trained with the following hyperparameters: {Batch size = 128, optimizer = AdamOptimizer with λ = 10 −4, total # of iterations=50,000.}The attack network has three hidden fully-connected layers of 300 units, trained with the following hyperparameters: {Batch size = 128, dropout rate = 0.5, optimizer = AdamOptimizer with 10 −3, total # of iterations=30,000.} For minimax, alt, and maximin optimization, the total number of iteration was 100,000. The sensitivity-penalty coefficient of γ = 1 was used in Alg. 1. We preprocess the CIFAR-10 dataset by removing the mean and normalizing the pixel values with the standard deviation of all pixels in the image. It is followed by clipping the values to ±2 standard deviations and rescaling to [−1, 1]. The architecture of the CIFAR classifier is similar to the Tensorflow model 3 but is simplified further by removing the local response normalization layers. With the simple structure, we attained ∼ 78% accuracy with the test data. The classifier is trained with the following hyperparameters: {Batch size = 128, optimizer = AdamOptimizer with λ = 10 −4, total # of iterations=100,000.}The attack network has three hidden fully-connected layers of 300 units, trained with the following hyperparameters: {Batch size = 128, dropout rate = 0.5, optimizer = AdamOptimizer with σ = 10 −3, total # of iterations=30,000.} For minimax, alt, and maximin optimization, the total number of iteration was 100,000. The sensitivity-penalty coefficient of γ = 1 was used in Alg. 1.In the rest of the appendix, we repeat all the experiments with the MNIST dataset using the CIFAR-10 dataset. Table 8: Error rates of different attacks on various adversarially-trained classifiers for CIFAR-10. FGSM-curr means the FGSM attack on the specific classifier on the leftmost column. Adv FGSM is the classifier adversally trained with FGSM attacks. Sens FGSM is the of minimizing the sensitivity penalty. LWA FGSM is the of minimizing FORMULA7 Table 9: Error rates of FGSM vs learning-based attack network (AttNet) on various adversariallytrained classifiers for CIFAR-10. FGSM-curr/AttNet-curr means they are computed/trained for the specific classifier on the leftmost column. Note that FGSM fails to attack against the'hardened' networks (Adv FGSM80 and Sens FGSM), but AttNet can still attack them successfully.
A game-theoretic solution to adversarial attacks and defenses.