DECK: Model Hardening for Defending Pervasive Backdoors
Pervasive backdoors are triggered by dynamic and pervasive input
perturbations. They can be intentionally injected by attackers or naturally
exist in normally trained models. They have a different nature from the
traditional static and localized backdoors that can be triggered by perturbing
a small input area with some fixed pattern, e.g., a patch with solid color.
Existing defense techniques are highly effective for traditional backdoors.
However, they may not work well for pervasive backdoors, especially regarding
backdoor removal and model hardening. In this paper, we propose a novel model
hardening technique against pervasive backdoors, including both natural and
injected backdoors. We develop a general pervasive attack based on an
encoder-decoder architecture enhanced with a special transformation layer. The
attack can model a wide range of existing pervasive backdoor attacks and
quantify them by class distances. As such, using the samples derived from our
attack in adversarial training can harden a model against these backdoor
vulnerabilities. Our evaluation on 9 datasets with 15 model structures shows
that our technique can enlarge class distances by 59.65% on average with less
than 1% accuracy degradation and no robustness loss, outperforming five
hardening techniques such as adversarial training, universal adversarial
training, MOTH, etc. It can reduce the attack success rate of six pervasive
backdoor attacks from 99.06% to 1.94%, surpassing seven state-of-the-art
backdoor removal techniques.