Previous work discusses and attempts to address problems with OOD issues typically present in ablation studies and feature removal based explanation methods in general [1]}, [2]}, [3]}. [1]} state that "without retraining it is unclear whether the degradation in model performance comes from the distribution shift or because the features that were removed are truly informative". However, we have observed a lack in consensus on the appropriate way to interpret fine-grained explanations yielded from retrained surrogate models. Although retraining-based methodologies diverge from the post-hoc paradigm, there are some fair criticisms to be made of ablation studies.
