Exploiting the Relationship Between Kendall's Rank Correlation and Cosine Similarity for Attribution Protection
Model attributions are important in deep neural networks as they aid
practitioners in understanding the models, but recent studies reveal that
attributions can be easily perturbed by adding imperceptible noise to the
input. The non-differentiable Kendall's rank correlation is a key performance
index for attribution protection. In this paper, we first show that the
expected Kendall's rank correlation is positively correlated to cosine
similarity and then indicate that the direction of attribution is the key to
attribution robustness. Based on these findings, we explore the vector space of
attribution to explain the shortcomings of attribution defense methods using
$\ell_p$ norm and propose integrated gradient regularizer (IGR), which
maximizes the cosine similarity between natural and perturbed attributions. Our
analysis further exposes that IGR encourages neurons with the same activation
states for natural samples and the corresponding perturbed samples, which is
shown to induce robustness to gradient-based attribution methods. Our
experiments on different models and datasets confirm our analysis on
attribution protection and demonstrate a decent improvement in adversarial
robustness.