Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.
Saliency is typically studied in the context of saliency modeling, in which a model outputs a saliency map for an image that matches human fixation data, or salient object segmentation @cite_41 . Separately, several works have considered extracting a saliency map for understanding classification network decisions @cite_43 @cite_40 . @cite_16 formulate a loss function that causes a student network to have similar "saliency" to a teacher network. They model saliency as a reduction operation @math applied to a volume of convolutional activations, which we adopt due to its simplicity. Here, we investigate using a saliency map for a downstream task. Recent work has begun to explore saliency maps as inputs for prominent object detection @cite_17 and image captioning @cite_27 , pointing to further uses of saliency-based vision models.
@cite_41 @cite_43 @cite_40 @cite_27 @cite_16 @cite_17
1894057436 2280306935 2605929543 2270657321
Visual saliency is a fundamental problem in both cognitive and computational sciences, including computer vision. In this paper, we discover that a high-quality visual saliency model can be learned from multiscale features extracted using deep convolutional neural networks (CNNs), which have had many successes in visual recognition tasks. For learning such saliency models, we introduce a neural network architecture, which has fully connected layers on top of CNNs responsible for feature extraction at three different scales. We then propose a refinement method to enhance the spatial coherence of our saliency results. Finally, aggregating multiple saliency maps computed for different levels of image segmentation can further boost the performance, yielding saliency maps better than those generated from a single segmentation. To promote further research and evaluation of visual saliency models, we also construct a new large database of 4447 challenging images and their pixelwise saliency annotations. Experimental results demonstrate that our proposed method is capable of achieving state-of-the-art performance on all public benchmarks, improving the F-Measure by 5.0 and 13.2 respectively on the MSRA-B dataset and our new dataset (HKU-IS), and lowering the mean absolute error by 5.7 and 35.1 respectively on these two datasets. Saliency is an attribute that is not included in an object itself, but arises from complex relations to the scene. Common belief in neuroscience is that objects are eye-catching if they exhibit an anomaly in some basic feature of human perception. This enables detection of object-like structures without prior knowledge. In this paper, we introduce an approach that models these object-to-scene relations based on probability theory. We rely on the conventional structure of cognitive visual attention systems, measuring saliency by local center to surround differences on several basic feature cues and multiple scales, but innovate how to model appearance and to quantify differences. Therefore, we propose an efficient procedure to compute ML-estimates for (multivariate) normal distributions of local feature statistics. Reducing feature statistics to Gaussians facilitates a closed-form solution for the W 2-distance (Wasserstein metric based on the Euclidean norm) between a center and a surround distribution. On a widely used benchmark for salient object detection, our approach, named CoDi-Saliency (for Continuous Distributions), outperformed nine state-of-the-art saliency detectors in terms of precision and recall. Image saliency detection has recently witnessed rapid progress due to deep convolutional neural networks. However, none of the existing methods is able to identify object instances in the detected salient regions. In this paper, we present a salient instance segmentation method that produces a saliency mask with distinct object instance labels for an input image. Our method consists of three steps, estimating saliency map, detecting salient object contours and identifying salient object instances. For the first two steps, we propose a multiscale saliency refinement network, which generates high-quality salient region masks and salient object contours. Once integrated with multiscale combinatorial grouping and a MAP-based subset optimization framework, our method can generate very promising salient object instance segmentation results. To promote further research and evaluation of salient instance segmentation, we also construct a new database of 1000 images and their pixelwise salient instance annotations. Experimental results demonstrate that our proposed method is capable of achieving state-of-the-art performance on all public benchmarks for salient region detection as well as on our new dataset for salient instance segmentation. Salient object detection increasingly receives attention as an important component or step in several pattern recognition and image processing tasks. Although a variety of powerful saliency models have been intensively proposed, they usually involve heavy feature (or model) engineering based on priors (or assumptions) about the properties of objects and backgrounds. Inspired by the effectiveness of recently developed feature learning, we provide a novel deep image saliency computing (DISC) framework for fine-grained image saliency computing. In particular, we model the image saliency from both the coarse-and fine-level observations, and utilize the deep convolutional neural network (CNN) to learn the saliency representation in a progressive manner. In particular, our saliency model is built upon two stacked CNNs. The first CNN generates a coarse-level saliency map by taking the overall image as the input, roughly identifying saliency regions in the global context. Furthermore, we integrate superpixel-based local context information in the first CNN to refine the coarse-level saliency map. Guided by the coarse saliency map, the second CNN focuses on the local context to produce fine-grained and accurate saliency map while preserving object details. For a testing image, the two CNNs collaboratively conduct the saliency computing in one shot. Our DISC framework is capable of uniformly highlighting the objects of interest from complex background while preserving well object details. Extensive experiments on several standard benchmarks suggest that DISC outperforms other state-of-the-art methods and it also generalizes well across data sets without additional training. The executable version of DISC is available online: http: vision.sysu.edu.cn projects DISC .