@inproceedings{
adebayo2023identifying,
title={Identifying regularization schemes that make feature attributions faithful},
author={Julius Adebayo and Samuel Don Stanton and Simon Kelow and Michael Maser and Richard Bonneau and Vladimir Gligorijevic and Kyunghyun Cho and Stephen Ra and Nathan C. Frey},
booktitle={NeurIPS 2023 Workshop on New Frontiers of AI for Drug Discovery and Development},
year={2023},
url={https://openreview.net/forum?id=YI85Jrvpj0}
}
Feature attribution methods assign a score to each input dimension as a measure of the relevance of that dimension to a model's output. Despite wide use, the feature importance rankings induced by gradient-based feature attributions are unfaithful, that is, they do not correlate with the input-perturbation sensitivity of the model---unless the model is trained to be adversarially robust. Here we demonstrate that these concerns translate to models trained for protein function prediction tasks. Despite making a model's gradient-based attributions faithful to the model, adversarial training has low real-data performance. We find that independent Gaussian noise corruption is an effective alternative, to adversarial training, that confers faithfulness onto a model's gradient-based attributions without performance degradation. On the other hand, we observe no meaningful faithfulness benefits from regularization schemes like dropout and weight decay. We translate these insights to a real-world protein function prediction task, where the gradient-based feature attributions of noise-regularized models, correctly indicate low sensitivity to irrelevant gap tokens in a protein's sequence alignment.
Identifying regularization schemes that make feature attributions faithful
Julius Adebayo Samuel Don Stanton Simon Kelow Michael Maser Richard Bonneau Vladimir Gligorijevic Kyunghyun Cho Stephen Ra Nathan C. Frey