A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> The primary problem dealt with in this paper is the following. Given some description of a visual object, find that object in an actual photograph. Part of the solution to this problem is the specification of a descriptive scheme, and a metric on which to base the decision of "goodness" of matching or detection. <s> BIB001 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> This paper describes a computational approach for accurately determining the location of human eyes in unconstrained monoscopic gray level images. The proposed method is based on exploiting the flow field characteristics that arise due to the presence of a dark iris surrounded by a light sclera. A novel aspect of the proposed method lies in its use of both spatial and temporal information to detect the location of the eyes. The spatial processing utilizes flow field information to select a pool of potential candidate locations for the eyes. Temporal processing uses the principle of continuity to filter out the actual location of the eyes from the pool of potential candidates. Extensions for gaze angle determination, and the tracking of human point-of-regard are indicated. <s> BIB002 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> Abstract We present a new approach for eye detection using the variance projection function. The variance projection function is developed and employed to locate landmarks of the human eye which are then used to guide the detection of the eye position and shape. A number of eye images are selected to evaluate the capability of the proposed method and the results are encouraging. <s> BIB003 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> In this paper we present a computationally efficient framework for part-based modeling and recognition of objects. Our work is motivated by the pictorial structure models introduced by Fischler and Elschlager. The basic idea is to represent an object by a collection of parts arranged in a deformable configuration. The appearance of each part is modeled separately, and the deformable configuration is represented by spring-like connections between pairs of parts. These models allow for qualitative descriptions of visual appearance, and are suitable for generic recognition problems. We address the problem of using pictorial structure models to find instances of an object in an image as well as the problem of learning an object model from training examples, presenting efficient algorithms in both cases. We demonstrate the techniques by learning models that represent faces and human bodies and using the resulting models to locate the corresponding objects in novel images. <s> BIB004 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> We present a method for object categorization in real-world scenes. Following a common consensus in the field, we do not assume that a figure-ground segmentation is available prior to recognition. However, in contrast to most standard approaches for object class recognition, our approach automatically segments the object as a result of the categorization. <s> BIB005 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> Automatic eye detection and tracking is an important component for advanced human-computer interface design. Accurate eye localization can help develop a successful system for face recognition and emotion identification. In this article, we propose a novel approach to detect and track eyes using geometric surface features on topographic manifold of eye images. First, in the joint spatial-intensity domain, a facial image is treated as a 3D terrain surface or image topographic manifold. In particular, eye regions exhibit certain intrinsic geometric traits on this topographic manifold, namely, the pit-labeled center and hillside-like surround regions. Applying a terrain classification procedure on the topographic manifold of facial images, each location of the manifold can be labeled to generate a terrain map. We use the distribution of terrain labels to represent the eye terrain pattern. The Bhattacharyya affinity is employed to measure the distribution similarity between two topographic manifolds. Based on the Bhattacharyya kernel, a support vector machine is applied for selecting proper eye pairs from the pit-labeled candidates. Second, given detected eyes on the first frame of a video sequence, a mutual-information-based fitting function is defined to describe the similarity between two terrain surfaces of neighboring frames. By optimizing the fitting function, eye locations are updated for subsequent frames. The distinction of the proposed approach lies in that both eye detection and eye tracking are performed on the derived topographic manifold, rather than on an original-intensity image domain. The robustness of the approach is demonstrated under various imaging conditions and with different facial appearances, using both static images and video sequences without background constraints. <s> BIB006 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> This paper presents a novel method for detecting and localizing objects of a visual category in cluttered real-world scenes. Our approach considers object categorization and figure-ground segmentation as two interleaved processes that closely collaborate towards a common goal. As shown in our work, the tight coupling between those two processes allows them to benefit from each other and improve the combined performance. ::: ::: The core part of our approach is a highly flexible learned representation for object shape that can combine the information observed on different training examples in a probabilistic extension of the Generalized Hough Transform. The resulting approach can detect categorical objects in novel images and automatically infer a probabilistic segmentation from the recognition result. This segmentation is then in turn used to again improve recognition by allowing the system to focus its efforts on object pixels and to discard misleading influences from the background. Moreover, the information from where in the image a hypothesis draws its support is employed in an MDL based hypothesis verification stage to resolve ambiguities between overlapping hypotheses and factor out the effects of partial occlusion. ::: ::: An extensive evaluation on several large data sets shows that the proposed system is applicable to a range of different object categories, including both rigid and articulated objects. In addition, its flexible representation allows it to achieve competitive object detection performance already from training sets that are between one and two orders of magnitude smaller than those used in comparable systems. <s> BIB007 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> The literature on the topic has shown a strong correlation between the degree of precision of face localization and the face recognition performance. Hence, there is a need for precise facial feature detectors, as well as objective measures for their evaluation and comparison. In this paper, we will present significant improvements to a previous method for precise eye center localization, by integrating a module for mouth localization. The technique is based on Support Vector Machines trained on optimally chosen Haar wavelet coefficients. The method has been tested on several public databases; the results are reported and compared according to a standard error measure. The tests show that the algorithm achieves high precision of localization. <s> BIB008 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-of-the-art results in the PASCAL object detection challenges. While deformable part models have become quite popular, their value had not been demonstrated on difficult benchmarks such as the PASCAL data sets. Our system relies on new methods for discriminative training with partially labeled data. We combine a margin-sensitive approach for data-mining hard negative examples with a formalism we call latent SVM. A latent SVM is a reformulation of MI--SVM in terms of latent variables. A latent SVM is semiconvex, and the training problem becomes convex once latent information is specified for the positive examples. This leads to an iterative training algorithm that alternates between fixing latent values for positive examples and optimizing the latent SVM objective function. <s> BIB009 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> This paper addresses the task of efficient object class detection by means of the Hough transform. This approach has been made popular by the Implicit Shape Model (ISM) and has been adopted many times. Although ISM exhibits robust detection performance, its probabilistic formulation is unsatisfactory. The PRincipled Implicit Shape Model (PRISM) overcomes these problems by interpreting Hough voting as a dual implementation of linear sliding-window detection. It thereby gives a sound justification to the voting procedure and imposes minimal constraints. We demonstrate PRISM's flexibility by two complementary implementations: a generatively trained Gaussian Mixture Model as well as a discriminatively trained histogram approach. Both systems achieve state-of-the-art performance. Detections are found by gradient-based or branch and bound search, respectively. The latter greatly benefits from PRISM's feature-centric view. It thereby avoids the unfavourable memory trade-off and any on-line pre-processing of the original Efficient Subwindow Search (ESS). Moreover, our approach takes account of the features' scale value while ESS does not. Finally, we show how to avoid soft-matching and spatial pyramid descriptors during detection without losing their positive effect. This makes algorithms simpler and faster. Both are possible if the object model is properly regularised and we discuss a modification of SVMs which allows for doing so. <s> BIB010 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> To detect multiple objects of interest, the methods based on Hough transform use non-maxima supression or mode seeking in order to locate and to distinguish peaks in Hough images. Such postprocessing requires tuning of extra parameters and is often fragile, especially when objects of interest tend to be closely located. In the paper, we develop a new probabilistic framework that is in many ways related to Hough transform, sharing its simplicity and wide applicability. At the same time, the framework bypasses the problem of multiple peaks identification in Hough images, and permits detection of multiple objects without invoking nonmaximum suppression heuristics. As a result, the experiments demonstrate a significant improvement in detection accuracy both for the classical task of straight line detection and for a more modern category-level (pedestrian) detection problem. <s> BIB011 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Exploiting structural information <s> We propose an efficient and generic facial feature localization method based on a weighted vector concentration approach. Our method does not require any specific priors on facial shape but implicitly learns its structural information from a training data. Unlike previous work, facial feature points are globally estimated by the concentration of directional vectors from sampling points on a face region, and those vectors are weighted by using local likelihood patterns which discriminate the appropriate position of the feature points. The directional vectors and local likelihood patterns are provided through nearest neighbor search between local patterns around the sampling points and a trained codebook of extended templates. The combination of the global vector concentration and the verification with the local likelihood patterns achieves robust facial feature point detection. We demonstrate that our method outperforms state-of-the-art method based on the Active Shape Models in our evaluation. <s> BIB012
Active shape model  , implicity shape model BIB005 , enhanced pictorial model  Typical methods to measure such patterns include Variance Projection Function (VPF) and Integral Projection Function (IPF) BIB003 , both of which are adopted in General Projection Function (GPF)  under a unified architecture for accurate eye localization. These projection functions actually estimate the global intensity distribution around the coarse eye region. However, in some real-world applications, the global intensity distribution might be deteriorated by noisy light spots of iris. To address this,  proposed to accumulate locally smoothed version of pixel intensity, which tends to be more stable compared to the global one. Alternatively, Wang et al. BIB006 developed a facial landscape navigation technique for eye localization, in which the interested intensity pattern (a pit at the eye center surrounded by hillside) is searched in a 3D terrain surface manifold of the face. In a similar method, gradient patterns BIB002 instead of intensity patterns are calculated around the eye region and served as the template for eye matching. Unlike photometric appearance features which capture the visual aspects of the eyes, the spatial topological features characterize the pattern of the eyes in a different way. In particular, the eye consists of components such as eyelids, iris and pupil, and these components have regular structural relationships. Such structural features are less affected by environmental conditions than appearance features. And clearly, the two types of features (i. e., appearance and structural features) are complementary to each other in depicting eye patterns. Since the eye is relatively smaller than the face, it is also convenient to model eyes structural features in the face context. Pictorial Structure Model (PS, BIB001 ) is a typical method in the line. The basic idea is to decompose an object as a set of parts and then use a graph structure to model the topological relationship between them. After a model like this is constructed, it can be used to localize the object of interest by measuring the appearance fitness and structural deformation simultaneously, where m i ðl i Þ is the similarity for part i at position l i according to appearance model, and d ij ðl i ; l j Þ measures the spatial structure confidence for part i and part j located at position l i and l j correspondingly. The challenge is how to solve this energy minimization problem efficiently so as to find the optimal configuration of parts positions L n . For this purpose, Felzenszwalb and Huttenlocher BIB004 presented an efficient method to automatically estimate the topological structure by simplifying the graph structure as a tree and restricting the form of connections between parts as a linear one rather than quadratic in the number of possible locations for each part (Fig. 10) . In a recent development BIB009 , they treat the locations of the parts as latent variables and learn them in a discriminative framework. This essentially reformulates the energy function of Eq. (1) by canceling the second term and merging it within the appearance model in the first term to make it more computational tractable. This method has been successfully applied to the task of object detection although in principle it can also be used for the problem of eye localization. To this end, Tan et al.  proposed a method which enhances the pictorial model in two aspects for the purpose of eye localization. First, to avoid estimating the multi-modal distribution of eye appearance under the complex real-life scenarios, a discriminative method is used to substitute the simple generative model used in BIB004 . This is in spirit similar to the latent SVM method in BIB009 , but the difference lies in that it is the label rather than the location of each part that is treated as a latent variable in  . This helps the model capture the meaning of each part (i.e., identifying them respectively as eyes, nose, etc.). Second, an improved structural description method is introduced to make it more robust against affine transformations as rotation, scale and translation. This method is tested on the challenging LFW data set  with promising results. Campadelli et al. BIB008 also proposed to use the information of other facial features (mouth in their case) as further constraints to reduce the number of false positives. Their methods are tested on several public databases (cf. Table 3) . Active Shape Model (ASM)  is another representative method to model the structure information of an object. In ASM, the shape of an object is described in terms of shape vector, which is a set of coordinates of these landmark points arranged in a predefined fixed order. The landmark points need to be manually annotated, usually on the contour and the key components of an object. ASM then uses these annotated training data to build a statistical model, so as to guide an iterative search on a test image for its shape vector which conforms to all the shape variations possible for this particular type of object (e.g., face). This model is suitable for locating multiple facial features simultaneously under high image resolution. One of the major advantages of structural methods lies in that it provides a nice mechanism to infer the location of an object by estimating the locations of its parts. Hough transform based methods have been successfully adapted to this purpose recently BIB010 BIB005 BIB007 BIB011 . Implicit Shape Model (ISM, BIB005 BIB007 ) is a typical votingbased structure model, in which the shape model is not explicitly constructed but represented loosely in terms of a bag of patches. Different from the usual bag of patches model, the key idea of ISM is to maintain a spatial occurrence distribution for each visual codeword, such that it can be used not only for the representation of local appearance but also for casting votes for possible positions of the object center as well. This Hough voting mechanism effectively bypasses the difficulty of optimization problem encountered in many structured methods (cf. Eq. (1)). Furthermore, it is robust against partial occlusion and large appearance variations. Recently, Barinova et al. BIB011 provided a new probabilistic formulation for Hough voting and Lehmann et al. BIB010 interpreted Hough voting as a dual implementation of linear sliding window detection, which leads to a fast implementation of ISM. Kozakaya et al. BIB012 proposed to improve the stableness of voting space using the dense sampling strategy.