Unifying Gradients to Improve Real-world Robustness for Deep Networks

The wide application of deep neural networks (DNNs) demands an increasing amount of attention to their real-world robustness, i.e., whether a DNN resists black-box adversarial attacks, among them score-based query attacks (SQAs) are the most threatening ones because of their practicalities and effectiveness: the attackers only need dozens of queries on model outputs to seriously hurt a victim network. Defending against SQAs requires a slight but artful variation of outputs due to the service purpose for users, who share the same output information with attackers. In this paper, we propose a real-world defense, called Unifying Gradients (UniG), to unify gradients of different data so that attackers could only probe a much weaker attack direction that is similar for different samples. Since such universal attack perturbations have been validated as less aggressive than the input-specific perturbations, UniG protects real-world DNNs by indicating attackers a twisted and less informative attack direction. To enhance UniG's practical significance in real-world applications, we implement it as a Hadamard product module that is computationally-efficient and readily plugged into any model. According to extensive experiments on 5 SQAs and 4 defense baselines, UniG significantly improves real-world robustness without hurting clean accuracy on CIFAR10 and ImageNet. For instance, UniG maintains a CIFAR-10 model of 77.80% accuracy under 2500-query Square attack while the state-of-the-art adversarially-trained model only has 67.34% on CIFAR10. Simultaneously, UniG greatly surpasses all compared baselines in clean accuracy and the modification degree of outputs. The code would be released.


INTRODUCTION
Despite the great success of deep neural networks (DNNs), they are highly vulnerable to adversarial examples (AEs, [17,39,47]), which can misguide a well-trained network into incorrect predictions through imperceptible perturbations on inputs. This weakness poses a serious threat on DNNs for the real-world applications such as autonomous driving [3], speech recognition [20], and medical diagnosis [7]. In these scenarios, both prediction results and output probabilities are provided to users for better decision or judgement, while training datasets and network parameters are hidden from users. In other words, without cyber attacks [36], which is another important topic but out of our scope, the attackers are only accessible to network outputs, as what the users can obtain. Therefore, defending real-world attacks requires attention to score-based query attacks (SQAs, [1,2,19,24,25]), a more practical kind of black-box attack compared to white-box attacks [6,17] and transfer-based attacks [12,23] that can successfully attack networks within dozens of queries of network outputs. Since SQAs are hundreds of times more efficient than decision-based query attacks (DQAs, [4,8]) that use model decisions only, the most noteworthy threat in realworld scenarios comes from SQAs attackers.
Currently, the most effective defense approaches against SQAs are adversarial training (AT, [18,32,40]) and randomness injection (RI, [5,13,16,22,30,35,43]). The former uses on-the-fly AEs as (a part of) training data, and the latter injects randomness into inputs [13,35,43], parameters [16,22,31], or features [30] to randomize DNNs' outputs to confuse attackers. These methods perform well in defending SQAs, but always hurt clean accuracy, and such depression of users limits their deployment in real-world applications. In this regard, the accuracy is one key factor. Besides, it is also crucial to maintain accurate output probabilities [3,7,20,48], which, however, is usually ignored by the existing defenses. An extreme example is as follows. If the network only outputs probabilities of 0/1, which means, the classification probabilities are discarded while the clean accuracy is kept, then SQAs degrade into DQAs because of the absence of classification confidence information. Under this circumstances, the robustness under SQAs can be significantly improved to the level of the robustness under DQAs. However, this simple defense is rarely used in practice, due to the catastrophic destroy on the output probabilities. In summary, a good defense method under SQAs should keep both clean accuracy and clean outputs as far as possible.  The plug-in Hadamard product module slightly modifies the forward feature (ˆ ≈ ) and totally distorts the backward gradient (ˆ ≠ ), whereˆ is a less threatening a ack direction. (b) The visualization of the distribution of ,ˆ and ,ˆ with t-SNE method. The feature distribution of our UniG model is consistent with that of the vanilla model, while the gradient distribution of our model is distorted to be completely different. (c) The numerical result of the robust accuracy under Square a ack (an excellent SQA) and the logit (output) difference on clean data for different defense approaches. UniG has both the best robustness and smallest logit difference compared with a AT or RI model. The target of this paper is to improve the robustness under SQAs with little influence on clean accuracy and outputs. From another perspective, we intend to mislead the attackers but simultaneously be friendly to users, which seems quite challenging since the users and the attackers share the same information-output probabilities. Our opportunity lies in the fact that the users utilize the forward information (output probabilities) while the attackers mainly use the backward information (gradients), which is obtained by visiting the victim network for multiple times. Since the attackers need plenty of outputs to deduce the backward information, slight modifications on forward outputs could accumulate to gradually influence the ultimate attack directions. Therefore, a direct defense idea comes to our mind: Can we distort the backward information through slightly modifying forward outputs to mislead a ackers? Following this idea, we insert a simple but designed module into a well-trained network, as shown in Figure 1(a), where we wantˆ ≈ ,ˆ ≠ andˆ to be a less threatening gradient direction than , i.e., the adversarial perturbation in the direction ofˆ performs worse in terms of misleading the network. The first demandˆ ≈ can be easily satisfied by imposing a consistency constraint ˆ − ≤ . The second oneˆ ≠ is somehow difficult from two aspects: i) maximizing the difference between andˆ is hard with numerous training data; ii) the gradient difference on training data does not mean the same difference on test data. The third one is closely related to the second demand, since they are both constraints onˆ . Therefore, considering the two demands together, we design a transductive approach [41,42] to overcome above obstacles and simultaneously weaken the attack effect ofˆ . Specifically, we propose to minimize the gradient variance of different inputs at test time to distort the backward information to mislead the attackers. Because of the direct use of test batch data (labelled by the DNN in a self-supervised manner), the generalization gap problem no longer exists. And the optimization of gradient variance somehow achieves the weak attack effect ofˆ that we pursue. More explanations will be shown later. In summary, we propose a defense approach which slightly modifies forward outputs to distort the attackers' backward information by unifying gradients of different inputs, called Unifying Gradients (UniG) method. Figure 1(b) demonstrates the feature distribution in an experiment, showing that our method keeps the distribution while distorts the feature gradient distribution into a more concentrated one. Further numerical results, see Figure 1(c), clearly illustrate that UniG performs the best in improving robustness and keeping clean outputs simultaneously, compared with the existing defenses. It is worth noticing that our approach can be directly applied at test time without re-training the whole network parameters like other transductive defenses [41,42]. Since there are only a few parameters in the designed module and one optimization iteration is sufficient in practice, the extra computation is negligible.
Here, we briefly explain two important issues of our method: i) why minimizing the gradient variance of different inputs can distort the backward information; ii) what kind of simple modules can achieve the slight forward modification and the backward gradient distortion simultaneously.
• The reason that unifying feature gradients can distort the backward information is as follows.
For different input samples, the corresponding feature gradients are naturally different for standard DNNs. This difference is then contained in the outputs of query data and captured by attackers to generate the specific attack direction for each sample. Once these feature gradients are unified, their specific attack information is distorted and then the attackers can only dig an universal attack direction (UAP, [46]), which is much weaker [45]. In this way, the backward information contained in our model's outputs is distorted and leads the attackers to weaker attack directions. • To achieve the slight forward modification and backward gradient distortion simultaneously, we design a simple and plug-in module which operates Hadamard product with input features ∈ × , where is the batch size and is the feature dimension. Denoting the parameter of our module as ∈ × , the output can be represented asˆ = • . If is close to the all-one matrix, slight modifications on forward outputs can be ensured. Now let us consider the backward propagation. Denoting the gradient of as , our aim is to minimize the variance of { } =1 , where represents the feature gradient of the -th input. We can analytically calculateˆ = • , where ∈ andˆ is the distorted gradient shown in Figure 1(a). Therefore, by optimizing , the variance ofˆ can be minimized, i.e., the attackers are prevented to get image specific information which is crucial for SQAs.
Extensive experiments on CIFAR10 [27] and ImageNet [15] will demonstrate the effectiveness of our method. Overall, our method improves robust accuracy under Square attack [2] by > 46.18% while keeping clean accuracy (94.26% on CIFAR10 and 78.47% on ImageNet) with slight output modifications. Amazingly, the performance of our vanilla-based UniG network even exceeds performance of state-of-the-art (SOTA) adversarially-trained (AT) networks [14] with higher clean accuracy. Furthermore, plugging in with AT networks, our method reaches up to 84.32% robust accuracy under Square attack (2500 queries) while the AT network only has 51.92% accuracy. More results under different attacks, networks and datasets are presented in the experiment section Experiment, which comprehensively verifies the effectiveness and practicability of the proposed UniG method.
The main contributions of this work are as follows.
• We propose a new perspective to defend against SQAs in real-world scenarios: slightly modifying forward outputs to distort the backward information , and consequently misleading attackers to weaker attack directions. • We propose a simple and plug-and-play method called UniG to achieve aforementioned modification on outputs, which distorts the backward information by unifying gradients of different inputs. • Extensive experiments are conducted to verify the effectiveness of our method against SOTA SQAs. On both CIFAR10 and ImageNet datasets, our method significantly improves the robustness under SQAs with no sacrifice on clean accuracy and very small changes on clean outputs.

RELATED WORK 2.1 Score-based ery A acks
Score-based query attacks (SQAs) can generate AEs using only network outputs and a few dozen queries, which makes them become the most threatening attack in real-world applications. The main idea of SQAs is to utilize output probabilities to observe the change of the attack margin loss, i.e., the top-1 probability subtracting the maximum probability of remaining classes, and to find an effective attack direction to minimize the margin loss. Current SQA algorithms can be divided into two categories according to how they find the attack direction: gradient estimation [1,9,24,25] and random search [2,19]. For gradient estimation, Zeroth Order Optimization (ZOO, [9]) firstly uses directional derivation [26] to estimate input gradients for attacking. Then, Natural Evolutionary Strategies (NES, [24]) is used in the condition of limited queries and information. Bandits [25] improves the query efficiency through utilizing data-dependent gradient prior information. In addition, SignHunter [1] reduces query budgets by ignoring gradient magnitudes and only estimating gradient signs. For random search methods, attackers randomly choose a direction and then add it into inputs for feedback outputs. Based on these outputs, attackers decide to keep this direction or abandon it. SimBA [19] is an early and simple random search based attack, which selects input perturbations from orthonormal basis. Square [2] is currently the SOTA SQA based on random search, which adds localized square-shape random noises into inputs. In addition to SQAs, there is another kind of query attacks named decision-based query attacks (DQAs, [4,8,11]) which only takes use of prediction results for attacking. Without doubt, DQAs are much more inefficient than SQAs because of their lack of probability information, i.e., thousands of queries are demanded to gradually mislead networks. This tremendous cost makes it hard to truly conduct in real-world scenarios. Therefore, we devote ourselves to defend only SQAs in this paper.

Adversarial Defense
Before, adversarial defenses are mainly designed to defend white-box attacks [6,32,39], among which adversarial training (AT [18,40]) is the most popular and comprehensive defense for its robustness under both white-box and black-box attacks. Recently, defenses for SQAs draw researchers' attention, since SQAs are very threatening in real-world applications. These defense approaches can be categorized into two types: adversarial detection and randomness injection. The first one utilizes the similarity between malicious query data [10,29,33] to improve the robustness. Specifically, they store history queries and compare them with the current query to detect adversarial examples. But this kind of defenses is more likely to be invalid when facing long-interval query and requires much more storage resources. As to the second type, defenses are designed to add random noise into different parts of models. For instance, [5] proposes to add random Gaussian noise into inputs to defend SQAs, and then [35] adopts it and gives a theoretical analysis for its defense effect. [22] and [30] respectively add trainable random Gaussian noise into model parameters and intermediate features for improving robustness, and [43] injects randomness through random data transformation. In addition, [13,28,38] propose to inject large random noises into network training processes for certified robustness. These methods perform well under SQAs, but inevitably reduce clean accuracy or suffer from a larger training cost. Apart from the above defenses designed for SQAs, recently proposed transductive methods [41,42] are also effective in defending SQAs, which dynamically optimize network parameters at test time to adapt attacks. However, this kind of defenses always faces with the problem of slow test speed because of their dynamic adjustments of network parameters. Different from the above methods, our proposed approach pays attention to the essence of SQAs, that is, the attacker explores backward information from forward outputs. Through deliberately modifying these outputs, we can distort the backward information to mislead the attackers into finding weaker attack directions.

Preliminaries and Motivation
Let us denote the victim model as : → , and the benign data as ( , ) ∈ ( , ). An attack aims to craft an adversarial example ′ which locates at the neighborhood of but misguides the model to an incorrect prediction class. The attack optimization problem can be summarized as follows, min ′ ∈ ( ) where ( ) = { ′ | − ′ ≤ } indicates the ball around with a radius , and denotes the predicted logits (or probabilities) of the -th class. An effective attack algorithm can make ( ′ ) ≤ 0 so that the predicted class is no longer the true label . In order to solve (1), the most common method is the projected gradient descent, which iteratively optimizes the objective function by gradients of ( ′ ) w.r.t. ′ .
For SQAs, the gradients can not be obtained directly since only the output logits of the victim model are accessible to attackers. To solve this problem, SQAs use direction derivation or some random search method to estimate gradients. The key of these methods is to utilize forward information of queries to infer backward information. Therefore, we propose a new defense idea that modifies the forward outputs slightly to prevent attackers from inferring correct backward information. Previous randomness injection methods [22,30,35] actually also have the above effect, but implicitly. The way is to add random noise into inputs, parameters or features at inference time so that the forward outputs of queries are also perturbed by random noises. The corresponding gradients estimated by attackers become inaccurate and then the final crafted AEs are more likely to be invalid. However, the injected noises are harmful to clean accuracy and unfriendly to users because of their randomness. In this paper, without output randomness, we explicitly optimize the proposed forward modification for the sake of distorting gradient information, which significantly improves the robustness under SQAs and meanwhile maintains clean accuracy with less output modifications.

UniG: Unifying Data's Gradients to Defend SQAs
The overall goal of our method can be summarized as follows: Our defense modelˆ shares similar clean outputs with the victim model , but distorts the vulnerable attack directions ( ) toˆ ( ) whose attack effect is weaker, where ( ) is the gradient of cross-entropy loss w.r.t . In this way, attackers craft AEs using the distorted attack direction ( ) and consequently the corresponding AEs are less threatening to both the victim model and our defense model.
To achieve the above goal, we design an optimization problem to generate our defense model as the following, whereˆ ( ) represents the gradients of cross-entropy loss w.r.t ( denotes the -th sample in a batch). The constraint obviously corresponds to the slight modification target, where controls the degree of output offset. The objective function corresponds to the goal of distorting the gradient information and making it less threatening. The explanation is as follows.
For a standard training model, the gradients of different inputs are naturally inconsistent with each other. The attackers utilize these gradients to generate a characteristic adversarial perturbation for each sample. The key is, if we enforce these characteristic adversarial perturbations to be the same with each other, then the attackers can only discover an universal adversarial perturbation (UAP), which is of course weaker than those image-dependent adversarial perturbations [45,46]. Therefore, in our method, as we explicitly constrain the gradient to be uniform across samples, the attackers can only obtain the UAP gradient information, losing the information of image-dependent gradients. Consequently, our model shows better robustness under SQAs by showing a weaker attack direction. Figure 2(a) and 2(b) verify the above statement experimentally. The weaker attack direction leads to the easy-classified feature distribution of the AEs of our model, as shown in Figure 2(a). And because the image-dependent attack directions are distorted, the margin loss rarely reduces with more queries, as Figure 2(b) shows.
To solve problem (3), a direct way is to finetune the victim model using the above loss function. However, this solution is not only resource-consuming, but also difficult because of millions of network parameters and training samples. Therefore, we propose an alternative solution. We replacê with a simple module which is inserted into the penultimate layer of the victim model, as Figure 1(a) shows. The optimization process of is done during inference time with current test batch inputs as the training data of problem 3. Noticing that test data have no labels, we utilize the prediction label as the true label in cross-entropy loss to calculateˆ . The overall problem becomes the following self-supervised formulation, which is easy to optimize, This problem converts (3) from a global parameter optimization problem into a local one, wherê ,ˆ , and respectively represents feature, feature gradients, and batch size, as Figure 1(a) shows. The operation of is designed to do Hadamard product with input features, that is,ˆ = • , which is simple but efficient. To keep forward outputs, every element of is expected to be close to one, while the objective loss in problem (4) gives an instructional direction to optimize around all-one matrix. As a result, the optimized modification on features plays an important role in distorting backward information to mislead attackers. We solve problem (4) using gradient descent (GD) algorithm, and the process of optimizing is integrated into the forward calculation of our defense model, which is shown in Alg. 1. The parameters of are dynamically optimized with different test batch data to ensure the gradient unity during inference time. Considering the circumstance that data batch size equals to one, we cascade training data with test inputs in this situation, which is acceptable in terms of computational overhead since only a few training samples, e.g., five, is needed. It is worth noticing that the objective loss is easy to optimize because only the linear layer and our designed module are involved in the calculation. Our experiment results show that with only one iteration step, the gradient variance can be efficiently minimized to significantly improve the robustness under SQAs.

EXPERIMENT 4.1 Setup
Datasets. We conduct our experiments on two widely-used classification datasets: CIFAR10 [27] and ImageNet [15]. For CIFAR10, the whole test dataset is used to evaluate method performances. For ImageNet, we randomly choose 1K images (one image for one class) from the validation set as test data. Models. PreResNet18 [21] and WideResNet-50-2 [44] are used as typical model architectures. On CIFAR10, PreResNet18 with vanilla training achieves 94.26% accuracy. On ImageNet, we directely use the pre-trained model from torchvision package of PyTorch [34], which has 78.47% accuracy. Score-based Query Attacks. We evaluate defense methods under five popular SQAs, including Square [2], SimBA [19], Sign [1], NES [24], and Bandits [25]. The first two are based on random search and the rest are based on gradient estimation. Most of the results in this paper are obtained under the untargeted ℓ ∞ norm attack, but we also present the results of the targeted and ℓ 2 norm attack in Section Performance under More Attacks for completeness. The ℓ ∞ attack bound is set as Calculate and min-max normalize the gradientˆ 5: Compute the objective loss ( , ) in problem (4) 6: Update with ← − · ( , ) 7: Clip with − 1 ∞ ≤ 8: end for 9: Compute final outputˆ = ( ( )) with optimized 10: returnˆ 8/255 for CIFAR10 and 4/255 for ImageNet, and the 2 attack bound is set as 1 for CIFAR10 and 5 for ImageNet. The query budget of SQAs is set to be 100 and 2500 for evaluating defense methods under different attack intensities. UniG setting. The UniG module is plugged into vanilla training models. For its parameters, we set =0.5, =1, =10 on CIFAR10 and =0.1, =1, =1 on ImageNet. Compared defenses. The proposed defense method will be compared with adversarial training (AT, [18,37]), random noise defense (RND, [35]), parameter noise injection (PNI, [22]), and dynamic inference (DENT, [41]) methods. The AT models are obtained from Robustbench 1 [14] and we choose the most robust one for comparison. Random noise defense, which adds random noise into inputs or outputs, is conducted with noise variance 0.02, as recommended by [35]. The model of PNI is obtained from the corresponding github code 2 and we choose the best for comparison. DENT is a transductive method and we insert it into the vanilla training model, as the same as UniG.
Metric. For a good defense, we need to consider three folds: the clean accuracy, the logits difference, and the robust accuracy, i.e., the remaining accuracy under attacks. The logits difference reflects the output difference between vanilla model and the current defense model, which is measured by the 2 norm in this paper. Other norms can be used as well and the conclusion is similar. The smaller it is, the more friendly the model is to users. Table 1 comprehensively reports the defense performance of the proposed UniG together with AT, RND, PNI, and DENT. AT could improve the robust accuracy under different attacks but it degrades clean accuracy by about 7% for CIFAR10 and 10% for ImageNet. The other defenses undertake effort to avoid the downgrade on clean accuracy, but the robust accuracy is generally lower than AT, especially on CIFAR10. By contrast, our UniG model achieves similar defense performance as AT, actually for most cases UniG is more robust than AT, and meanwhile greatly remains the clean accuracy, almost the same as the vanilla training. For logits difference, the advantage of UniG is more significant that the logits difference is generally reduced by an order of magnitude, which means UniG is much more user-friendly than the other defenses. Table 1. The comparison of different defense methods under ∞ norm a acks (#query = 100/2500) on CIFAR10 and ImageNet. The clean accuracy, logits difference and robust accuracy are reported. The higher the clean accuracy and robust accuracy, the smaller the logits difference, indicating the be er performance of the defense method.. As a plug-and-play module, UniG can be readily applied in any network, for example in a model trained by adversarial examples. The performance of UniG in AT can be found in Table 2. From AT, the classification accuracy of UniG-AT is well kept on clean examples and only drops 3.04% on CI-FAR10 under 2500-query Square attack, while the AT model drops about 20%. Similar performance could be found on both CIFAR10 and ImageNet under different attacks.

Performance under More A acks
Before, we have shown the performance under current popular SQAs in an untargeted setting and a ℓ ∞ -norm bound. In this section, we further evaluate the performance of our method under other settings, i.e., ℓ 2 / ℓ ∞ norm and target (-T) / untarget (-UT) attacks as listed in the first row of Table 3. We adopt Square attack, the current SOTA SQA attack, for robustness evaluation. As Table 3 verified, plugging UniG into vanilla and AT model can both achieve significant robustness improvement with a little clean accuracy drop. For instance, on CIFAR10, our UniG-Vanilla model improves robust accuracy by no less than 71.29% under all attack settings (2500 queries), and UniG-AT improves by ≥ 13.99% compared with AT model, while our method keeps the same or even higher clean accuracy.

Hyper-parameter Study
Previous experiments are conducted using the fixed hyper-parameters as introduced before. To further evaluate our method, we here study the influence of different hyper-parameters on the performance of our method. The main hyper-parameters include the forward constraint parameter , the optimization iteration , and the learning rate . The parameter decides the largest elementwise difference between the parameters of our module and all-one matrix, which controls the trade-off between the clean performance and the robustness of our model. The iteration and the learning rate are related to the optimization process of our module . For fast test speed, we adopt one iteration and relatively bigger to optimize in the previous experiments, while in this section, we adopt different and to observe their influence on the clean accuracy, the logits difference and the robustness under Square attack. As Fig 3(a) shows, with the increase of , the logits difference slightly increases, but is still less than which of other methods. And for the optimization iteration , we observe the same phenomenon as , see Fig 3(b). For the change of , the robustness keeps stable, while the clean accuracy suffers from small fluctuations and the logits difference slightly increases, see Fig 3(c). Apart from the aforementioned hyper-parameters, we also evaluate the performance for different batch sizes, since our objective loss is dependent on the input test data. The result in Figure 3(d) reveals that our method performs better with a larger batch size, but even with the batch size equals to 32, the robustness performance is quite good.

Other Algorithmic Discussions
This section discusses the computational complexity and the adaptive attack of our method. The first one is related to the time and space overhead of our method, and the second one is frequently used to test the worst-case robustness of a defense approach. Computational Complexity. Because of the optimization process contained in our model's forward calculation, the computational complexity of our approach inevitably increases. There are two metrics to measure this complexity -the number of model parameters and the number of Floating Point Operations (FLOPs) of the forward process. The first one reflects the space overhead and the second one represents the time overhead. Table 4 reports the computational complexity of Vanilla, DENT and UniG model on CIFAR10 with the PreResNet18 architecture. Although there is some growth in the number of network parameters, the FLOPs of our method is far less than that of DENT, with a slight increase compared with that of the vanilla model.   Adaptive Attack. In general, an adaptive attack refers to an elaborately designed white-box attack with full knowledge of the defense strategy. Nevertheless, in the real-world cases, the attacker and the victim model are double-blind to each other, which means the defense strategy is not prior information for the attacker. Therefore, a reasonable adaptive attack in realistic situations is that the attacker firstly employs extra queries to deduce the defense strategy, and then does adaptive attack with normal queries, which seems less effective than normal SQAs because it is hard to conjecture defense strategies. Back to the adaptive attack of UniG, since our method distorts the image-dependent attack directions and only shows the attacker an universal attack direction, the only adaptive attack we come up with is to force the attack direction to be diverse on different inputs. However, even ignoring the strategy conjecture process, such adaptive attack would dramatically increase the query budget because of the diversity constraint. As we know, an important reason for the menace of SQAs in real-world applications is that they can successfully attack the victim model with dozens of queries. Generally, there is no obvious adaptive strategy to enhance SQAs for the proposed UniG. But of course no system is impenetrable and then the proposed UniG needs to be further advanced with SQAs that will emerge in the future.

CONCLUSION
In this paper, we propose a new defense method named Unifying Gradients (UniG) to defend the most threatening attack in real applications-score-based query attacks (SQAs). The proposed method is based on the idea that the forward outputs are slightly modified to distort the backward information and ultimately mislead the attacker to find weaker attack directions. To achieve it in real cases, we propose to insert a Hadamard product operation module into the penultimatelayer of a well-trained network, and optimize its parameters with a designed self-supervised loss.
With comprehensive experiments on CIFAR10 and ImageNet, it is verified that our approach can significantly boost the robustness under SQAs with no sacrifice of clean accuracy and a few variation on clean outputs. Noticing that the designed module is plug-and-play with negligible extra computational overhead, the overall method has a promising application prospect in the future.