Recently the NLP community has witnessed a resurgence in interpreting and explaining deep neural network based models [1]}, [2]}, [3]}. Most of the interpretation based methods involve one of the following ways of interpreting models: a) sample oriented interpretations: where the interpretation is based on changes in the prediction score with either upweighting or perturbing samples [1]}, [2]}, [6]}; b) interpretations based on feature attributions using attention or input perturbation or gradient-based measures; [7]}, [8]}, [9]}; c) interpretations using surrogate linear models [10]} – these methods can provide local interpretations based on input samples or features. However, the presence of inherent randomness makes it difficult to accurately interpret deep neural models among other forms of pathologies [8]}.
