Circumventing Backdoor Defenses That Are Based on Latent Separability
Deep learning models are vulnerable to backdoor poisoning attacks. In
particular, adversaries can embed hidden backdoors into a model by only
modifying a very small portion of its training data. On the other hand, it has
also been commonly observed that backdoor poisoning attacks tend to leave a
tangible signature in the latent space of the backdoored model i.e. poison
samples and clean samples form two separable clusters in the latent space.
These observations give rise to the popularity of latent separability
assumption, which states that the backdoored DNN models will learn separable
latent representations for poison and clean populations. A number of popular
defenses (e.g. Spectral Signature, Activation Clustering, SCAn, etc.) are
exactly built upon this assumption. However, in this paper, we show that the
latent separation can be significantly suppressed via designing adaptive
backdoor poisoning attacks with more sophisticated poison strategies, which
consequently render state-of-the-art defenses based on this assumption less
effective (and often completely fail). More interestingly, we find that our
adaptive attacks can even evade some other typical backdoor defenses that do
not explicitly build on this separability assumption. Our results show that
adaptive backdoor poisoning attacks that can breach the latent separability
assumption should be seriously considered for evaluating existing and future
defenses.