A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> We describe experiments with eigenfaces for recognition and interactive search in a large-scale face database. Accurate visual recognition is demonstrated using a database of O(10/sup 3/) faces. The problem of recognition under general viewing orientation is also examined. A view-based multiple-observer eigenspace technique is proposed for use in face recognition under variable pose. In addition, a modular eigenspace description technique is used which incorporates salient features such as the eyes, nose and mouth, in an eigenfeature layer. This modular representation yields higher recognition rates as well as a more robust framework for face recognition. An automatic feature extraction technique using feature eigentemplates is also demonstrated. > <s> BIB001 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> We present a neural network-based upright frontal face detection system. A retinally connected neural network examines small windows of an image and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We present a straightforward procedure for aligning positive face examples for training. To collect negative examples, we use a bootstrap algorithm, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting nonface training examples, which must be chosen to span the entire space of nonface images. Simple heuristics, such as using the fact that faces rarely overlap in images, can further improve the accuracy. Comparisons with several other state-of-the-art face detection systems are presented, showing that our system has comparable performance in terms of detection and false-positive rates. <s> BIB002 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Eye feature location is an important step in automatic visual interpretation and human face recognition. In this paper, a novel approach for locating eye centers in face areas under probabilistic framework is devised. After grossly locating a face, we first find the areas which left and right eyes lies in. Then an appearance-based eye detector is used to detect the possible left and right eye separately. According to their probabilities, the candidates are subsampled to merge those in near positions. Finally, the remaining left and right eye candidates are paired; each possible eye pair is normalized and verified. According to their probabilities, the precise eye positions are decided. The experimental results demonstrate that our method can effectively cope with different eye variations and achieve better location performance on diverse test sets than some newly proposed methods. And the influence of precision of eye location on face recognition is also probed. The location of other face organs such as mouth and nose can be incorporated in the framework easily. <s> BIB003 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> This paper presents a novel approach for eye detection using a hierarchy cascade classifier based on Adaboost statistical learning method combined with SVM (Support Vector Machines) post classifier. On the first stage a face detector is used to locate the face in the whole image. After finding the face, an eye detector is used to detect the possible eye candidates within the face areas. Finally, the precise eye positions are decided by the eye-pair SVM classifiers which using geometrical and relative position information of eye-pair and the face. Experimental results show that this method can effectively cope with various image conditions and achieve better location performance on diverse test sets than some newly proposed methods. <s> BIB004 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> The accuracy of face alignment affects the performance of a face recognition system. Since face alignment is usually conducted using eye positions, an accurate eye localization algorithm is therefore essential for accurate face recognition. In this paper, we first study the impact of eye locations on face recognition accuracy, and then introduce an automatic technique for eye detection. The performance of our automatic eye detection technique is subsequently validated using FRGC 1.0 database. The validation shows that our eye detector has an overall 94.5% eye detection rate, with the detected eyes very close to the manually provided eye positions. In addition, the face recognition performance based on the automatic eye detection is shown to be comparable to that of using manually given eye positions. <s> BIB005 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> We address the task of accurately localizing the eyes in face images extracted by a face detector, an important problem to be solved because of the negative effect of poor localization on face recognition accuracy. We investigate three approaches to the task: a regression approach aiming to directly minimize errors in the predicted eye positions, a simple Bayesian model of eye and non-eye appearance, and a discriminative eye detector trained using AdaBoost. By using identical training and test data for each method we are able to perform an unbiased comparison. We show that, perhaps surprisingly, the simple Bayesian approach performs best on databases including challenging images, and performance is comparable to more complex state-of-the-art methods. <s> BIB006 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> We present a method for precise eye localization that uses two Support Vector Machines trained on properly selected Haar wavelet coefficients. The evaluation of our technique on many standard databases exhibits very good performance. Furthermore, we study the strong correlation between the eye localization error and the face recognition rate. <s> BIB007 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Straightforward classification using kernelized SVMs requires evaluating the kernel for a test vector and each of the support vectors. For a class of kernels we show that one can do this much more efficiently. In particular we show that one can build histogram intersection kernel SVMs (IKSVMs) with runtime complexity of the classifier logarithmic in the number of support vectors as opposed to linear for the standard approach. We further show that by precomputing auxiliary tables we can construct an approximate classifier with constant runtime and space requirements, independent of the number of support vectors, with negligible loss in classification accuracy on various tasks. This approximation also applies to 1 - chi2 and other kernels of similar form. We also introduce novel features based on a multi-level histograms of oriented edge energy and present experiments on various detection datasets. On the INRIA pedestrian dataset an approximate IKSVM classifier based on these features has the current best performance, with a miss rate 13% lower at 10-6 False Positive Per Window than the linear SVM detector of Dalal & Triggs. On the Daimler Chrysler pedestrian dataset IKSVM gives comparable accuracy to the best results (based on quadratic SVM), while being 15times faster. In these experiments our approximate IKSVM is up to 2000times faster than a standard implementation and requires 200times less memory. Finally we show that a 50times speedup is possible using approximate IKSVM based on spatial pyramid features on the Caltech 101 dataset with negligible loss of accuracy. <s> BIB008 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Most successful object recognition systems rely on binary classification, deciding only if an object is present or not, but not providing information on the actual object location. To perform localization, one can take a sliding window approach, but this strongly increases the computational cost, because the classifier function has to be evaluated over a large set of candidate subwindows. In this paper, we propose a simple yet powerful branch-and-bound scheme that allows efficient maximization of a large class of classifier functions over all possible subimages. It converges to a globally optimal solution typically in sublinear time. We show how our method is applicable to different object detection and retrieval scenarios. The achieved speedup allows the use of classifiers for localization that formerly were considered too slow for this task, such as SVMs with a spatial pyramid kernel or nearest neighbor classifiers based on the chi2-distance. We demonstrate state-of-the-art performance of the resulting systems on the UIUC Cars dataset, the PASCAL VOC 2006 dataset and in the PASCAL VOC 2007 competition. <s> BIB009 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> In this paper, we describe a novel type of feature for fast and accurate face detection. The feature is called Locally Assembled Binary (LAB) Haar feature. LAB feature is basically inspired by the success of Haar feature and Local Binary Pattern (LBP) for face detection, but it is far beyond a simple combination. In our method, Haar features are modified to keep only the ordinal relationship (named by binary Haar feature) rather than the difference between the accumulated intensities. Several neighboring binary Haar features are then assembled to capture their co-occurrence with similar idea to LBP. We show that the feature is more efficient than Haar feature and LBP both in discriminating power and computational cost. Furthermore, a novel efficient detection method called feature-centric cascade is proposed to build an efficient detector, which is developed from the feature-centric method. Experimental results on the CMU+MIT frontal face test set and CMU profile test set show that the proposed method can achieve very good results and amazing detection speed. <s> BIB010 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Automatic facial feature localization has been a long-standing challenge in the field of computer vision for several decades. This can be explained by the large variation a face in an image can have due to factors such as position, facial expression, pose, illumination, and background clutter. Support Vector Machines (SVMs) have been a popular statistical tool for facial feature detection. Traditional SVM approaches to facial feature detection typically extract features from images (e.g. multiband filter, SIFT features) and learn the SVM parameters. Independently learning features and SVM parameters might result in a loss of information related to the classification process. This paper proposes an energy-based framework to jointly perform relevant feature weighting and SVM parameter learning. Preliminary experiments on standard face databases have shown significant improvement in speed with our approach. <s> BIB011 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Sliding window classifiers are among the most successful and widely applied techniques for object localization. However, training is typically done in a way that is not specific to the localization task. First a binary classifier is trained using a sample of positive and negative examples, and this classifier is subsequently applied to multiple regions within test images. We propose instead to treat object localization in a principled way by posing it as a problem of predicting structured data: we model the problem not as binary classification, but as the prediction of the bounding box of objects located in images. The use of a joint-kernelframework allows us to formulate the training procedure as a generalization of an SVM, which can be solved efficiently. We further improve computational efficiency by using a branch-and-bound strategy for localization during both training and testing. Experimental evaluation on the PASCAL VOC and TU Darmstadt datasets show that the structured training procedure improves performance over binary training as well as the best previously published scores. <s> BIB012 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> We present methods for training high quality object detectors very quickly. The core contribution is a pair of fast training algorithms for piece-wise linear classifiers, which can approximate arbitrary additive models. The classifiers are trained in a max-margin framework and significantly outperform linear classifiers on a variety of vision datasets. We report experimental results quantifying training time and accuracy on image classification tasks and pedestrian detection, including detection results better than the best previous on the INRIA dataset with faster training. <s> BIB013 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> This paper introduces a class of correlation filters called average of synthetic exact filters (ASEF). For ASEF, the correlation output is completely specified for each training image. This is in marked contrast to prior methods such as synthetic discriminant functions (SDFs) which only specify a single output value per training image. Advantages of ASEF training include: insensitivity to over-fitting, greater flexibility with regard to training images, and more robust behavior in the presence of structured backgrounds. The theory and design of ASEF filters is presented using eye localization on the FERET database as an example task. ASEF is compared to other popular correlation filters including SDF, MACE, OTF, and UMACE, and with other eye localization methods including Gabor Jets and the OpenCV cascade classifier. ASEF is shown to outperform all these methods, locating the eye to within the radius of the iris approximately 98.5% of the time. <s> BIB014 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> This paper addresses the task of efficient object class detection by means of the Hough transform. This approach has been made popular by the Implicit Shape Model (ISM) and has been adopted many times. Although ISM exhibits robust detection performance, its probabilistic formulation is unsatisfactory. The PRincipled Implicit Shape Model (PRISM) overcomes these problems by interpreting Hough voting as a dual implementation of linear sliding-window detection. It thereby gives a sound justification to the voting procedure and imposes minimal constraints. We demonstrate PRISM's flexibility by two complementary implementations: a generatively trained Gaussian Mixture Model as well as a discriminatively trained histogram approach. Both systems achieve state-of-the-art performance. Detections are found by gradient-based or branch and bound search, respectively. The latter greatly benefits from PRISM's feature-centric view. It thereby avoids the unfavourable memory trade-off and any on-line pre-processing of the original Efficient Subwindow Search (ESS). Moreover, our approach takes account of the features' scale value while ESS does not. Finally, we show how to avoid soft-matching and spatial pyramid descriptors during detection without losing their positive effect. This makes algorithms simpler and faster. Both are possible if the object model is properly regularised and we discuss a modification of SVMs which allows for doing so. <s> BIB015 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Eye localization is an important part in face recognition system, because its precision closely affects the performance of face recognition. Although various methods have already achieved high precision on the face images with high quality, their precision will drop on low quality images. In this paper, we propose a robust eye localization method for low quality face images to improve the eye detection rate and localization precision. First, we propose a probabilistic cascade (P-Cascade) framework, in which we reformulate the traditional cascade classifier in a probabilistic way. The P-Cascade can give chance to each image patch contributing to the final result, regardless the patch is accepted or rejected by the cascade. Second, we propose two extensions to further improve the robustness and precision in the P-Cascade framework. There are: (1) extending feature set, and (2) stacking two classifiers in multiple scales. Extensive experiments on JAFFE, BioID, LFW and a self-collected video surveillance database show that our method is comparable to state-of-the-art methods on high quality images and can work well on low quality images. This work supplies a solid base for face recognition applications under unconstrained or surveillance environments. <s> BIB016 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> This paper presents a new eye localization method via Multiscale Sparse Dictionaries (MSD). We built a pyramid of dictionaries that models context information at multiple scales. Eye locations are estimated at each scale by fitting the image through sparse coefficients of the dictionary. By using context information, our method is robust to various eye appearances. The method also works efficiently since it avoids sliding a search window in the image during localization. The experiments in BioID database prove the effectiveness of our method. <s> BIB017 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> Many classifiers are trained with massive training sets only to be applied at test time on data from a different distribution. How can we rapidly and simply adapt a classifier to a new test distribution, even when we do not have access to the original training data? We present an on-line approach for rapidly adapting a “black box” classifier to a new test data set without retraining the classifier or examining the original optimization criterion. Assuming the original classifier outputs a continuous number for which a threshold gives the class, we reclassify points near the original boundary using a Gaussian process regression scheme. We show how this general procedure can be used in the context of a classifier cascade, demonstrating performance that far exceeds state-of-the-art results in face detection on a standard data set. We also draw connections to work in semi-supervised learning, domain adaptation, and information regularization. <s> BIB018 </s> A literature survey on robust and efficient eye localization in real-life scenarios <s> Statistical appearance models <s> In this paper, we take a look at an enhanced approach for eye detection under difficult acquisition circumstances such as low-light, distance, pose variation, and blur. We present a novel correlation filter based eye detection pipeline that is specifically designed to reduce face alignment errors, thereby increasing eye localization accuracy and ultimately face recognition accuracy. The accuracy of our eye detector is validated using data derived from the Labeled Faces in the Wild (LFW) and the Face Detection on Hard Datasets Competition 2011 (FDHD) sets. The results on the LFW dataset also show that the proposed algorithm exhibits enhanced performance, compared to another correlation filter based detector, and that a considerable increase in face recognition accuracy may be achieved by focusing more effort on the eye localization stage of the face recognition process. Our results on the FDHD dataset show that our eye detector exhibits superior performance, compared to 11 different state-of-the-art algorithms, on the entire set of difficult data without any per set modifications to our detection or preprocessing algorithms. The immediate application of eye detection is automatic face recognition, though many good applications exist in other areas, including medical research, training simulators, communication systems for the disabled, and automotive engineering. <s> BIB019
Even armed with a proper feature representation which accounts for certain variability associated with target occurrences, one still needs to build a final classification stage that can handle residual variability and learn effective models from relatively few training samples. This section describes some popular methods for this purpose. Popular classifiers are constructed in either a generative or a discriminative way  . The generative methods try to recover the class conditional probability distribution of eyes and use it to see how likely a testing patch is generated from it. Due to the high dimensionality of eye patches, the conditional density is best estimated on a lower dimensional manifold, which can be found using manifold learning methods like Locally Linear Embedding (LLE) or Principal Component Analysis (PCA) BIB001 . Generative methods: a successful generative method is the probabilistic PCA model for object representation proposed by Moghaddam and Pentaland  , in which the training samples are first projected into their column space using PCA, then a Gaussian density is estimated there to model the class conditional distribution of positive samples. Evergingham and Zisserman BIB006 extended this idea to include non-eye model pðxjeÞ. Then the prediction of a new patch x can be performed by looking at the margin (or log-likelihood ratio) between the outputs of the two models given x : llrðxÞ ¼ log pðxjeÞ−log pðxjeÞ, where both eye model pðxjeÞ and non-eye model pðxjeÞ are assumed to be Gaussian. Even though more complicated generative model, e.g., Gaussian Mixture Model, could be applied, this simple method yields very good localization performance on several databases consistently, which clearly illustrates the power of generative methods in this challenging task. To ensure accurate distribution estimation, however, a relatively large number of labeled training samples are desired. Discriminative methods: on the other hand, discriminative methods aim to find the discriminant function between eye and non-eye classes directly, in the form of decision surface, separating hyperplane, or threshold function. In this way, the problem of eye localization boils down to a binary classification problem. Typical classifiers for this include Support Vector Machine (SVM) BIB006 BIB004  , AdaBoost  BIB005  BIB003 , neural networks BIB002 , and so on. Among others, SVM is widely used in practice due to its ease of use and its good generalization capability BIB004  . However, in many cases a good generalization performance is only guaranteed with enough number of support vectors coupled with nonlinear kernel function, which may significantly increase the computational costs and memory when testing. However for certain specific kernels, the computation could be made very efficient BIB008 as well as the training method BIB013 . Alternatively, it is possible to adopt an efficient branch-and-bound optimization strategy to reduce the number of matching without sacrificing the accuracy of classifiers BIB009 . Another way to achieve a good tradeoff between accuracy and efficiency is the boosted cascade of features  BIB003 BIB005   BIB010 , which adopts the 'coarse-to-fine' strategy to select a bunch of discriminative features and use them to construct more powerful classifiers increasingly. The testing step is very efficient since each level of the cascaded classifiers consist of a linear combination of a few simple weak classifiers and only those patches with high likelihood will be passed on for further examination. To enable reliable feature selection, sometimes the candidate patches are set to include both eyes  BIB003 . To deal with the problem of eye localization in low quality face images, BIB016 introduced a quality adaptive cascade that works in a probabilistic framework (P-Cascade), which allows all image patches to contribute to the final result with some probability. Optimizing the localization accuracy: compared to generative methods, discriminative methods are more efficient in exploiting different types of visual features from samples without assuming Fig. 7 . Examples of multi-scale dictionaries BIB017 . Fig. 8 . Wavelet decomposition and feature selection for hierarchical SVM eye model BIB007 . any dependence between them, but they may not behave so robustly against the problem of probability drifting as their generative counterparts. The reason for this is that for the discriminative methods only the area of decision boundary is important while the interior distribution of each class is completely ignored by the model. Evergingham and Zisserman BIB006 compared several classifiers including both generative and discriminative ones and found that the simple Bayesian classifier outperforms several discriminative methods including SVM and AdaBoost method. One possible explanation for this is that the discriminative methods are sensitive to small localization errors in the ground truth BIB006 . Moreover, since the objective of the discriminative method is set to optimize the classification accuracy rather than localization accuracy, the trained classifier may not give the maximal response at the right object location. There are several approaches to address this problem. The simplest one is to explicitly tell the classifier which training examples are confusing, i.e., they look like but are actually not the target of interest. This can be done by the careful construction of the negative set. Those examples can either be sampled around random points in the iris neighborhood specified beforehand BIB011 , or be generated in a boosting fashion by identifying them using the current classifier  . Another natural way to explicitly incorporate the positions of the eyes is to formulate the task of localization as a regression rather than a binary classification problem. In this setting, the training data are given by a set of input images and their corresponding eye positions, and the goal is to learn a regressor which maps from the input image to the predicted eye position (in some invariant coordinate system, cf. BIB015 ). Different regression methods can be adopted according to how to specify the eye localization. In BIB014 , each training image is assigned a correlation image which is synthetically generated with a bright peak at the center of the target (e.g., the left eye) and small values everywhere else. This correlation image gives not only the locations of the eyes but also the corresponding desired response to be regressed. The regression is implemented by producing a correlation filter that exactly transforms each input image to its correlation image (see Fig. 9 ). The localization is performed by correlating a test image with the learned filter and selecting the global maximum in the correlation outputs. This testing procedure is very efficient without resorting to the time-consuming sliding window search. A similar method is adopted by Hefin et al. BIB019 . To deal with the ambiguity from multiple peak responses, they warp the face image using the candidate response pairs and select the one that leads to a face image with best quality BIB019 . The above filter methods implicitly learn a regression model for eye localization. Alternatively, as proposed in BIB006 , one can directly construct a linear regression model in the Hilbert space from an input image to its eye localization, i.e., a 2D coordinates in the image. To ignore the irrelevant variation, a square region centered at the mean over the training images of the ground truth eye positions is extracted for regressing. This method is shown to work better than SVM and AdaBoost-based methods, especially on the uncontrolled WWW images BIB006 . Moving forward with this line of research, Blaschko and Lampert BIB012 proposed a general object localization method which specifically optimizes localization accuracy within a large margin regression framework. In particular, instead of representing the positions of the eyes as 2D coordinates, a bounding box around the eye region, which is regarded as a more flexible and useful structure to be regressed in the output space, is adopted. With the help of joint kernel map, the training procedure is formulated under the framework of structured predicting  . This method is shown to boost the accuracy of object localization and has the potential to handle partial detections by properly evaluating the usefulness of image regions that contain portions of the object. Besides structured regression, Gaussian process can also be used for the purpose of object localization in general and eye localization in particular BIB018 .