Capturing the interesting components of an image is a key aspect of image understanding. When a speaker annotates an image, selecting labels that are informative greatly depends on the prior knowledge of a prospective listener. Motivated by cognitive theories of categorization and communication, we present a new unsupervised approach to model this prior knowledge and quantify the informativeness of a description. Specifically, we compute how knowledge of a label reduces uncertainty over the space of labels and utilize this to rank candidate labels for describing an image. While the full estimation problem is intractable, we describe an efficient algorithm to approximate entropy reduction using a tree-structured graphical model. We evaluate our approach on the open-images dataset using a new evaluation set of 10K ground-truth ratings and find that it achieves 65 agreement with human raters, largely outperforming other unsupervised baseline approaches.
* Image importance and object saliency. The problem of deciding which components in an image are important has been studied intensively. The main approaches involved identifying characteristics of objects and images that could contribute to importance, and use labeled data for predicting object importance. Elazary and Itti @cite_8 considered the order of object naming in the LabelMe dataset @cite_7 as a measure of the interest of an object and compare that to salient locations predicted by computational models of bottom-up attention. The elegant work of Spain and Perona @cite_18 examined which factors can predict the order in which objects will be mentioned given an image. @cite_19 characterized factors related to semantics, to composition and to the likelihood of attribute-object, and investigated how these affected the measures of importance. @cite_6 focused on predicting entry-level classes using a supervised approach. These studies also make it clear that the object saliency is strongly correlated with its perceived importance @cite_16 @cite_4 .
@cite_18 @cite_4 @cite_7 @cite_8 @cite_6 @cite_19 @cite_16
2107200795 937276653 1954873805 2165947725
We propose a principled probabilistic formulation of object saliency as a sampling problem. This novel formulation allows us to learn, from a large corpus of unlabelled images, which patches of an image are of the greatest interest and most likely to correspond to an object. We then sample the object saliency map to propose object locations. We show that using only a single object location proposal per image, we are able to correctly select an object in over 42 of the images in the Pascal VOC 2007 dataset, substantially outperforming existing approaches. Furthermore, we show that our object proposal can be used as a simple unsupervised approach to the weakly supervised annotation problem. Our simple unsupervised approach to annotating objects of interest in images achieves a higher annotation accuracy than most weakly supervised approaches. Recently, saliency detection has been becoming a popular topic in computer vision. In this paper we propose an object-level saliency detection algorithm which explicitly explores bottom-up visual attention and objectness cues. Firstly, some category-independent object candidates are segmented from the image by the quantized color attributes. Then two metrics, global cues and candidate objectness, are developed to estimate the saliency in the whole image and the object candidates respectively. We use global cues to describe the focusness and spatial distribution of color attributes in the entire image. As supplementary, candidate objectness can reveal the objectness of the object candidates. Finally, we integrate the two saliency cues to derive a saliency map of the image. By explicitly fusing candidate objectness and global cues, our proposed method is more suitable for processing images with complex background. Experimental results on three large benchmark databases demonstrate that the proposed method achieves more favorable performance than 15 recent state-of-the-art methods in terms of three evaluation criterions. We propose an object-level salient detection algorithm which explicitly explores bottom-up visual attention and objectness cues.Some category-independent object candidates are firstly segmented from the image by the quantized color attributes of images.Global cues and candidate objectness are developed, to evaluate bottom-up visual attention of the whole image and the objectness of the object candidates respectively.By explicitly combining local objectness cue with global low-level saliency cues with candidates location and color attributes, our proposed method is more suitable for processing images with complex background. People can immediately and precisely identify that an image contains 1, 2, 3 or 4 items by a simple glance. The phenomenon, known as Subitizing, inspires us to pursue the task of Salient Object Subitizing (SOS), i.e. predicting the existence and the number of salient objects in a scene using holistic cues. To study this problem, we propose a new image dataset annotated using an online crowdsourcing marketplace. We show that a proposed subitizing technique using an end-to-end Convolutional Neural Network (CNN) model achieves significantly better than chance performance in matching human labels on our dataset. It attains 94 accuracy in detecting the existence of salient objects, and 42–82 accuracy (chance is 20 ) in predicting the number of salient objects (1, 2, 3, and 4+), without resorting to any object localization process. Finally, we demonstrate the usefulness of the proposed subitizing technique in two computer vision applications: salient object detection and object proposal. How do we decide which objects in a visual scene are more interesting? While intuition may point toward high-level object recognition and cognitive processes, here we investigate the contributions of a much simpler process, low-level visual saliency. We used the LabelMe database (24,863 photographs with 74,454 manually outlined objects) to evaluate how often interesting objects were among the few most salient locations predicted by a computational model of bottom-up attention. In 43 of all images the model’s predicted most salient location falls within a labeled region (chance 21 ). Furthermore, in 76 of the images (chance 43 ), one or more of the top three salient locations fell on an outlined object, with performance leveling off after six predicted locations. The bottom-up attention model has neither notion of object nor notion of semantic relevance. Hence, our results indicate that selecting interesting objects in a scene is largely constrained by low-level visual properties rather than solely determined by higher cognitive processes.