Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attacks
The score-based query attacks (SQAs) pose practical threats to deep neural
networks by crafting adversarial perturbations within dozens of queries, only
using the model's output scores. Nonetheless, we note that if the loss trend of
the outputs is slightly perturbed, SQAs could be easily misled and thereby
become much less effective. Following this idea, we propose a novel defense,
namely Adversarial Attack on Attackers (AAA), to confound SQAs towards
incorrect attack directions by slightly modifying the output logits. In this
way, (1) SQAs are prevented regardless of the model's worst-case robustness;
(2) the original model predictions are hardly changed, i.e., no degradation on
clean accuracy; (3) the calibration of confidence scores can be improved
simultaneously. Extensive experiments are provided to verify the above
advantages. For example, by setting $\ell_\infty=8/255$ on CIFAR-10, our
proposed AAA helps WideResNet-28 secure $80.59\%$ accuracy under Square attack
($2500$ queries), while the best prior defense (i.e., adversarial training)
only attains $67.44\%$. Since AAA attacks SQA's general greedy strategy, such
advantages of AAA over 8 defenses can be consistently observed on 8
CIFAR-10/ImageNet models under 6 SQAs, using different attack targets and
bounds. Moreover, AAA calibrates better without hurting the accuracy. Our code
would be released.