Saliency methods can aid understanding of deep neural networks. Recent years have witnessed many improvements to saliency methods, as well as new ways for evaluating them. In this paper, we 1) present a novel region-based attribution method, XRAI, that builds upon integrated gradients ( 2017), 2) introduce evaluation methods for empirically assessing the quality of image-based saliency maps (Performance Information Curves (PICs)), and 3) contribute an axiom-based sanity check for attribution methods. Through empirical experiments and example results, we show that XRAI produces better results than other saliency methods for common models and the ImageNet dataset.
Recently, the reliability and validity of saliency methods have been critically examined. In particular, researchers have discovered that many saliency maps are fragile against adversarial attacks @cite_15 . @cite_3 introduced a set of sanity checks for saliency methods to ensure they produce valid results. Specifically, they introduced two general classes of sanity checks: similarity metrics and visualizations. In applying these checks, they found that some saliency methods produce similar outputs regardless of whether a model is trained or completely random. This result indicates that some saliency methods' results cannot be considered reliable explanations for a model's behavior. In the context of this paper's research, they found that Integrated Gradients pass the similarity metrics sanity check, but did not convincingly pass the visualization sanity check. As we show later, our technique introduces some key modifications to Integrated Gradients that enables it to successfully pass this sanity check.
@cite_15 @cite_3
1993939318 2519610629 2486803733 2442791716
Previous saliency detection algorithms used to focus on low level features directly or utilize a bunch of sample images and manually labeled ground truth to train a high level learning model. In this paper, we propose a novel coding-based saliency measure by exploring both global and local cues for saliency computation. Firstly, we construct a bottom-up saliency map by considering global contrast information via low level features. Secondly, by using a locality-constrained linear coding algorithm, a top-down saliency map is formulated based on the reconstruction error. To better exploit the local and global information, we integrate the bottom-up and top-down maps as the final saliency map. Extensive experimental results on three large benchmark datasets demonstrate that the proposed approach outperforms 22 state-of-the-art methods in terms of three popular evaluation measures, i.e., the Precision and Recall curve, Area Under ROC Curve and F-measure value. Furthermore, the proposed coding-based method can be easily applied in other methods for significant improvement. HighlightsWe present a coding-based algorithm for salient object detection.Integration of local and global cues makes the saliency maps more accurate, intact.Bottom-up maps provide foreground and background codebooks for following steps.Fusion of FC and BC based results makes the saliency results more uniform, robust.Our coding-based method can be easily applied in other methods for improvement. In this paper, we deal with a weakly supervised semantic segmentation problem where only training images with image-level labels are available. We propose a weakly supervised semantic segmentation method which is based on CNN-based class-specific saliency maps and fully-connected CRF. To obtain distinct class-specific saliency maps which can be used as unary potentials of CRF, we propose a novel method to estimate class saliency maps which improves the method proposed by (2014) significantly by the following improvements: (1) using CNN derivatives with respect to feature maps of the intermediate convolutional layers with up-sampling instead of an input image; (2) subtracting the saliency maps of the other classes from the saliency maps of the target class to differentiate target objects from other objects; (3) aggregating multiple-scale class saliency maps to compensate lower resolution of the feature maps. After obtaining distinct class saliency maps, we apply fully-connected CRF by using the class maps as unary potentials. By the experiments, we show that the proposed method has outperformed state-of-the-art results with the PASCAL VOC 2012 dataset under the weakly-supervised setting. In recent years, the object-level saliency detection has attracted much research attention, due to its usefulness in many high-level tasks. Existing methods are mostly based on the contrast hypothesis, which regards the regions with high contrast in a certain context as salient objects. Although the contrast hypothesis is effective in many scenarios, it cannot handle some difficult cases. As a remedy to address the weakness of contrast hypothesis, we propose a novel compactness hypothesis, which assumes salient regions are more compact than background from the perspectives of both color layout and texture layout. Based on the compactness hypotheses, we implement an effective object-level saliency detection method. In the proposed method, we first construct a weak saliency map based on the compact hypotheses, then collect samples from the weak saliency map to train a dedicated classifier. This classifier is applied on each individual pixel of the input image to produce a confidence score. Finally, the confidence scores are used to form a saliency map. This process is carried out at different scales, and the corresponding results are integrated into the formation of the final saliency map. The proposed approach is evaluated on eight benchmark data sets, where it delivers the competitive performance compared with the state-of-the-art methods. Top-down saliency models produce a probability map that peaks at target locations specified by a task goal such as object detection. They are usually trained in a supervised setting involving annotations of objects. We propose a weakly supervised top-down saliency framework using only binary labels that indicate the presence absence of an object in an image. First, the probabilistic contribution of each image patch to the confidence of an ScSPM-based classifier produces a Reverse-ScSPM (R-ScSPM) saliency map. Neighborhood information is then incorporated through a contextual saliency map which is estimated using logistic regression learnt on patches having high R-ScSPM saliency. Both the saliency maps are combined to obtain the final saliency map. We evaluate the performance of the proposed weakly supervised top-down saliency and achieves comparable performance with fully supervised approaches. Experiments are carried out on 5 challenging datasets across 3 different applications.