	Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation.
	 Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a single aggregated explanation. The aggregation is more robust and aligns better with the neural network than any single explanation method..
	 Second, we propose a new approach to evaluating explanation methods that circumvents the need for manual evaluation and is not reliant on the alignment of neural networks and humans decision processes. Despite the great success of neural networks especially in classic visual recognition problems, explaining the networks' decisions remains an open research problem Samek et al. (2019) . This is due in part to the complexity of the visual recognition problem and in part to the basic 'ill-posedness' of the explanation task. This challenge is amplified by the fact that there is no agreement on what a sufficient explanation is and how to evaluate an explanation method. Many different explanation strategies and methods have been proposed (Simonyan et al., 2013; Zeiler & Fergus, 2014; Bach et al., 2015; Selvaraju et al., 2017; Smilkov et al., 2017; Sundararajan et al., 2017) . Focusing on visual explanations for individual decisions, most methods either use a backpropagation approach or aim to construct a simpler linear model with an intuitive explanation. The plethora of explanation approaches is a signature of the high-level epistemic uncertainty of the explanation task. This paper is motivated by a key insight in machine learning: Ensemble models can reduce both bias and variance compared to applying a single model. A related approach was pursued for functional visualization in neuroimaging (Hansen et al., 2001 ). Here we for the first time explore the potential of aggregating explanations of individual visual decisions in reducing epistemic uncertainty for neural networks. We test the hypothesis that ensembles of multiple explanation methods are more robust than any single method. This idea is analyzed theoretically and evaluated empirically. We discuss the properties of the aggregate explanations and provide visual evidence that they combine features, hence are more complete and less biased than individual schemes. Based on this insight, we propose two ways to aggregate explanation methods, AGG-Mean and AGG-Var. In experiments on Imagenet, MNIST, and FashionMNIST, the aggregates identify relevant parts of the image more accurately than any single method. Second, we introduce IROF (Iterative Removal Of Features) as a new approach to quantitatively evaluate explanation methods without relying on human evaluation. We circumvent the problems of high correlation between neighbor pixels as well as the human bias that are present in current evaluation methods. In this work we gave a simple proof that aggregating explanation methods will perform at least as good as the typical individual method. In practice, we found evidence that aggregating methods outperforms any single method. We found this evidence substantiated across quantitative metrics. While our results show that different vanilla explanation methods perform best on different network architectures, an aggregation supersedes all of them on any given architecture. Additionally we proposed a novel way of evaluation for explanation methods that circumvents the problem of high correlation between pixels and does not rely on visual inspection by humans, an inherently misleading metric.
We show in theory and in practice that combining multiple explanation methods for DNN benefits the explanation.
{"src": "scitldr-AIC_train", "paper_id": "B1xeZJHKPB"}