Hardening DNNs against Transfer Attacks during Network Compression using Greedy Adversarial Pruning
The prevalence and success of Deep Neural Network (DNN) applications in
recent years have motivated research on DNN compression, such as pruning and
quantization. These techniques accelerate model inference, reduce power
consumption, and reduce the size and complexity of the hardware necessary to
run DNNs, all with little to no loss in accuracy. However, since DNNs are
vulnerable to adversarial inputs, it is important to consider the relationship
between compression and adversarial robustness. In this work, we investigate
the adversarial robustness of models produced by several irregular pruning
schemes and by 8-bit quantization. Additionally, while conventional pruning
removes the least important parameters in a DNN, we investigate the effect of
an unconventional pruning method: removing the most important model parameters
based on the gradient on adversarial inputs. We call this method Greedy
Adversarial Pruning (GAP) and we find that this pruning method results in
models that are resistant to transfer attacks from their uncompressed
counterparts.