Task-agnostic Defense against Adversarial Patch Attacks
Adversarial patch attacks mislead neural networks by injecting adversarial
pixels within a designated local region. Patch attacks can be highly effective
in a variety of tasks and physically realizable via attachment (e.g. a sticker)
to the real-world objects. Despite the diversity in attack patterns,
adversarial patches tend to be highly textured and different in appearance from
natural images. We exploit this property and present PatchZero, a task-agnostic
defense against white-box adversarial patches. Specifically, our defense
detects the adversarial pixels and "zeros out" the patch region by repainting
with mean pixel values. We formulate the patch detection problem as a semantic
segmentation task such that our model can generalize to patches of any size and
shape. We further design a two-stage adversarial training scheme to defend
against the stronger adaptive attacks. We thoroughly evaluate PatchZero on the
image classification (ImageNet, RESISC45), object detection (PASCAL VOC), and
video classification (UCF101) datasets. Our method achieves SOTA robust
accuracy without any degradation in the benign performance.