
We expand on the work done in [1]} to gain insight in the behaviour of different existing metrics, as well as some newly proposed ones.

We propose a new metric based on Sensitivity-n [2]}, and empirically show that it provides results with a higher signal-to-noise ratio on high-dimensional datasets.

We find that the performance of some methods is complementary to that of other methods, suggesting that a combination of these attribution methods may be valuable.

We find that, depending on the dataset, methods with strong theoretical foundations such as DeepSHAP [3]} do not necessarily outperform their computationally cheaper counterparts such as DeepLIFT [4]}. This suggests that a benchmarking approach can be useful to check if a computationally intensive method is truly more valuable than a simpler one for a given use case.

Finally, we provide general guidelines to perform such a benchmark for a given use case.

