Can Backdoor Attacks Survive Time-Varying Models?
Backdoors are powerful attacks against deep neural networks (DNNs). By
poisoning training data, attackers can inject hidden rules (backdoors) into
DNNs, which only activate on inputs containing attack-specific triggers. While
existing work has studied backdoor attacks on a variety of DNN models, they
only consider static models, which remain unchanged after initial deployment.
  In this paper, we study the impact of backdoor attacks on a more realistic
scenario of time-varying DNN models, where model weights are updated
periodically to handle drifts in data distribution over time. Specifically, we
empirically quantify the "survivability" of a backdoor against model updates,
and examine how attack parameters, data drift behaviors, and model update
strategies affect backdoor survivability. Our results show that one-shot
backdoor attacks (i.e., only poisoning training data once) do not survive past
a few model updates, even when attackers aggressively increase trigger size and
poison ratio. To stay unaffected by model update, attackers must continuously
introduce corrupted data into the training pipeline. Together, these results
indicate that when models are updated to learn new data, they also "forget"
backdoors as hidden, malicious features. The larger the distribution shift
between old and new training data, the faster backdoors are forgotten.
Leveraging these insights, we apply a smart learning rate scheduler to further
accelerate backdoor forgetting during model updates, which prevents one-shot
backdoors from surviving past a single model update.