Instead of computing the mean of saliency map distribution by averaging, Lapuschkin et al. [1]} separate the modes of class-specific saliency maps for classification by clustering them. By analyzing one of the clusters, they discovered that the model relied on a particular watermark in its predictions (revealing a flaw in the PASCAL VOC dataset[2]}). In our experiments (on MS-COCO) with clustering, we did not find any such anomalous examples, however we were able to observe vertically symmetric saliency maps (e.g., Fig. REF ) assigned into separate clusters.
