This paper presents a novel fixation prediction and saliency modeling framework based on inter-image similarities and ensemble of Extreme Learning Machines (ELM). The proposed framework is inspired by two observations, (1) the contextual information of a scene along with low-level visual cues modulates attention, (2) the influence of scene memorability on eye movement patterns caused by the resemblance of a scene to a former visual experience. Motivated by such observations, we develop a framework that estimates the saliency of a given image using an ensemble of extreme learners, each trained on an image similar to the input image. That is, after retrieving a set of similar images for a given image, a saliency predictor is learnt from each of the images in the retrieved image set using an ELM, resulting in an ensemble. The saliency of the given image is then measured in terms of the mean of predicted saliency value by the ensembles members.
Learning-based techniques are a large group of methods which are establishing a relation between a feature space and human fixations. For example, @cite_44 uses a nonlinear transformation to associate image patches with human eye movement statistics. @cite_60 , a linear SVM classifier is used to establish a relation between three channels of low- (intensity, color, etc), mid- (horizon line) and high-level (faces and people) features and human eye movements in order to produce a saliency map. In a similar vein, @cite_1 employs multiple-instance learning. By learning a classifier, @cite_23 @cite_21 estimate the optimal weights for fusing several conspicuity maps from observers' eye movement data. These approaches often learn a probabilistic classifier to determine the probability of a feature being salient. Then, they employ the estimated saliency probability in order to build a saliency map.
@cite_60 @cite_21 @cite_1 @cite_44 @cite_23
2134557905 2519610629 2546490537 2028087473
We assess the applicability of several popular learning methods for the problem of recognizing generic visual categories with invariance to pose, lighting, and surrounding clutter. A large dataset comprising stereo image pairs of 50 uniform-colored toys under 36 azimuths, 9 elevations, and 6 lighting conditions was collected (for a total of 194,400 individual images). The objects were 10 instances of 5 generic categories: four-legged animals, human figures, airplanes, trucks, and cars. Five instances of each category were used for training, and the other five for testing. Low-resolution grayscale images of the objects with various amounts of variability and surrounding clutter were used for training and testing. Nearest neighbor methods, support vector machines, and convolutional networks, operating on raw pixels or on PCA-derived features were tested. Test error rates for unseen object instances placed on uniform backgrounds were around 13 for SVM and 7 for convolutional nets. On a segmentation recognition task with highly cluttered images, SVM proved impractical, while convolutional nets yielded 16 7 error. A real-time version of the system was implemented that can detect and classify objects in natural scenes at around 10 frames per second. In this paper, we deal with a weakly supervised semantic segmentation problem where only training images with image-level labels are available. We propose a weakly supervised semantic segmentation method which is based on CNN-based class-specific saliency maps and fully-connected CRF. To obtain distinct class-specific saliency maps which can be used as unary potentials of CRF, we propose a novel method to estimate class saliency maps which improves the method proposed by (2014) significantly by the following improvements: (1) using CNN derivatives with respect to feature maps of the intermediate convolutional layers with up-sampling instead of an input image; (2) subtracting the saliency maps of the other classes from the saliency maps of the target class to differentiate target objects from other objects; (3) aggregating multiple-scale class saliency maps to compensate lower resolution of the feature maps. After obtaining distinct class saliency maps, we apply fully-connected CRF by using the class maps as unary potentials. By the experiments, we show that the proposed method has outperformed state-of-the-art results with the PASCAL VOC 2012 dataset under the weakly-supervised setting. Creation and selection of relevant features for machine learning applications (including image classification) is typically a process requiring significant involvement of domain knowledge. It is thus desirable to cover at least part of that process with semi-automated techniques capable of discovering and visualizing those geometric characteristics of images that are potentially relevant to the classification objective. In this work, we propose to utilize multi-scale singular value decomposition (MSVD) along with approximate nearest neighbors algorithm: both have been recently realized using the randomized approach, and can be efficiently run on large, high-dimensional datasets (sparse or dense). We apply this technique to create a multi-scale view of every point in a publicly available set of LIDAR data of riparian images, with classification objective being separating ground from vegetation. We perform “centralized MSVD” for every point and its neighborhood generated by an approximate nearest neighbor algorithm. After completion of this procedure, the original set of 3-dimensional data is augmented by 36 dimensions generated by MSVD (in three different scales), which is then processed using a novel discretization pre-processing method and the SVM classification algorithm with RBF kernel. The result is two times better that the one previously obtained (in terms of its classification error level). The generic nature of the MSVD mechanism and standard mechanisms used for classification (SVM) suggest a wider utility of the proposed approach for other problems as well. In this paper, we present a novel model for saliency prediction under a unified framework of feature integration. The model distinguishes itself by directly learning from natural images and automatically incorporating higher-level semantic information in a scalable manner for gaze prediction. Unlike most existing saliency models that rely on specific features or object detectors, our model learns multiple stages of features that mimic the hierarchical organization of the ventral stream in the visual cortex and integrate them by adapting their weights based on the ground-truth fixation data. To accomplish this, we utilize a multi-layer sparse network to learn low-, mid- and high-level features from natural images and train a linear support vector machine (SVM) for weight adaption and feature integration. Experimental results show that our model could learn high-level semantic features like faces and texts and can perform competitively among existing approaches in predicting eye fixations.