@inproceedings{
reber2023whats,
title={What's your Use Case? A Taxonomy of Causal Evaluations of Post-hoc Interpretability},
author={David Reber and Cristina Garbacea and Victor Veitch},
booktitle={Causal Representation Learning Workshop at NeurIPS 2023},
year={2023},
url={https://openreview.net/forum?id=xOgWq35OH1}
}
Post-hoc interpretability of neural network models, including Large Language Models (LLMs), often aims for mechanistic interpretations — detailed, causal descriptions of model behavior. However, human interpreters may lack the capacity or willingness to formulate intricate mechanistic models, let alone evaluate them. This paper addresses this challenge by introducing a taxonomy which dissects the overarching goal of mechanistic interpretability into constituent claims, each requiring distinct evaluation methods. By doing so, we transform these evaluation criteria into actionable learning objectives, providing a data-driven pathway to interpretability. This framework enables a methodologically rigorous yet pragmatic approach to evaluating the strengths and limitations of various interpretability tools.
What's your Use Case? A Taxonomy of Causal Evaluations of Post-hoc Interpretability
David Reber Cristina Garbacea Victor Veitch