A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> One of the main limitations of current remote eye gaze tracking (REGT) techniques is that the user's head must remain within a very limited area in front of the monitor screen. In this paper, we present a free head motion REGT technique. By projecting a known rectangular pattern of lights, the technique estimates the gaze position relative to this rectangle using an invariant property of projective geometry. We carry extensive analysis of similar methods using an eye model to compare their accuracy. Based on these results, we propose a new estimation procedure that compensates the angular difference between the eye visual axis and optical axis. We have developed a real time (30 fps) prototype using a single camera and 5 light sources to generate the light pattern. Experimental results shows that the accuracy of the system is about 1deg of visual angle <s> BIB001 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> A statistical active appearance model (AAM) is developed to track and detect eye blinking. The model has been designed to be robust to variations of head pose or gaze. In particular we analyze and determine the model parameters which encode the variations caused by blinking. This global model is further extended using a series of sub-models to enable independent modeling and tracking of the two eye regions. Several methods to enable measurement and detection of eye-blink are proposed and evaluated. The results of various tests on different image databases are presented to validate each model. <s> BIB002 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> In this paper, a new method for eye gaze tracking is proposed under natural head movement. In this method, Local-Binary-Pattern Texture Feature (LBP) is adopted to calculate the eye features according to the characteristic of the eye, and a precise Binocular Vision approach is used to detect the space coordinate of the eye. The combined features of space coordinates and LBP features of the eyes are fed into Support Vector Regression (SVR) to match the gaze mapping function, in the hope of tracking gaze direction under natural head movement. The experimental results prove that the proposed method can determine the gaze direction accurately. <s> BIB003 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> In recent years, research on human-computer interaction is becoming popular, most of which uses body movements, gestures or eye gaze direction. Until now, gazing estimation is still an active research domain. We propose an efficient method to solve the problem of the eye gaze point. We first locate the eye region by modifying the characteristics of the Active Appearance Model (AAM). Then by employing the Support Vector Machine (SVM), we estimate the five gazing directions through classification. The original 68 facial feature points in AAM are modified into 36 eye feature points. According to the two-dimensional coordinates of feature points, we classify different directions of eye gazing. The modified 36 feature points describe the contour of eyes, iris size, iris location, and the position of pupils. In addition, the resolution of cameras does not affect our method to determine the direction of line of sight accurately. The final results show the independence of classifications, less classification errors, and more accurate estimation of the gazing directions. <s> BIB004 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> We show that dually supervised manifold embedding can improve the performance of machine learning based person-independent and thus calibration-free gaze estimation. For this purpose, we perform a manifold embedding for each person in the training dataset and then learn a linear transformation that aligns the individual, person-dependent manifolds. We evaluate the effect of manifold alignment on the recently presented Columbia dataset, where we analyze the influence on 6 regression methods and 8 feature variants. Using manifold alignment, we are able to improve the person-independent gaze estimation performance by up to 31.2 % compared to the best approach without manifold alignment. <s> BIB005 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> In this work, we develop an appearance-based gaze tracking system allowing user to move their head freely. The main difficulty of the appearance-based gaze tracking method is that the eye appearance is sensitive to head orientation. To overcome the difficulty, we propose a 3-D gaze tracking method combining head pose tracking and appearance-based gaze estimation. We use a random forest approach to model the neighbor structure of the joint head pose and eye appearance space, and efficiently select neighbors from the collected high dimensional data set. Li-optimization is then used to seek for the best solution for regression from the selected neighboring samples. Experiment results shows that it can provide robust binocular gaze tracking results with less constraints but still provides moderate estimation accuracy of gaze estimation. <s> BIB006 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> In this paper we demonstrate efficient methods for continuous estimation of eye gaze angles with application to sign language videos. The difficulty of the task lies on the fact that those videos contain images with low face resolution since they are recorded from distance. First, we proceed to the modeling of face and eyes region by training and fitting Global and Local Active Appearance Models (LAAM). Next, we propose a system for eye gaze estimation based on a machine learning approach. In the first stage of our method, we classify gaze into discrete classes using GMMs that are based either on the parameters of the LAAM, or on HOG descriptors for the eyes region. We also propose a method for computing gaze direction angles from GMM log-likelihoods. We qualitatively and quantitatively evaluate our methods on two sign language databases and compare with a state of the art geometric model of the eye based on LAAM landmarks, which provides an estimate in direction angles. Finally, we further evaluate our framework by getting ground truth data from an eye tracking system Our proposed methods, and especially the GMMs using LAAM parameters, demonstrate high accuracy and robustness even in challenging tasks. <s> BIB007 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> This paper presents an accurate and efficient eye detection method using the discriminatory Haar features (DHFs) and a new efficient support vector machine (eSVM). The DHFs are extracted by applying a discriminating feature extraction (DFE) method to the 2D Haar wavelet transform. The DFE method is capable of extracting multiple discriminatory features for two-class problems based on two novel measure vectors and a new criterion in the whitened principal component analysis (PCA) space. The eSVM significantly improves the computational efficiency upon the conventional SVM for eye detection without sacrificing the generalization performance. Experiments on the Face Recognition Grand Challenge (FRGC) database and the BioID face database show that (i) the DHFs exhibit promising classification capability for eye detection problem; (ii) the eSVM runs much faster than the conventional SVM; and (iii) the proposed eye detection method achieves near real-time eye detection speed and better eye detection performance than some state-of-the-art eye detection methods. A discriminating feature extraction (DFE) method for two-class problems is proposed.The DFE method is applied to derive the discriminatory Haar features (DHFs) for eye detection.An efficient support vector machine (eSVM) is proposed to improve the efficiency of the SVM.An accurate and efficient eye detection method is presented using the DHFs and the eSVM. <s> BIB008 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> Appearance-based gaze estimation is believed to work well in real-world settings, but existing datasets have been collected under controlled laboratory conditions and methods have been not evaluated across multiple datasets. In this work we study appearance-based gaze estimation in the wild. We present the MPIIGaze dataset that contains 213,659 images we collected from 15 participants during natural everyday laptop use over more than three months. Our dataset is significantly more variable than existing ones with respect to appearance and illumination. We also present a method for in-the-wild appearance-based gaze estimation using multimodal convolutional neural networks that significantly outperforms state-of-the art methods in the most challenging cross-dataset evaluation. We present an extensive evaluation of several state-of-the-art image-based gaze estimation algorithms on three current datasets, including our own. This evaluation provides clear insights and allows us to identify key research challenges of gaze estimation in the wild. <s> BIB009 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> Estimating three-dimensional (3D) human eye gaze by capturing a single eye image without active illumination is challenging. Although the elliptical iris shape provides a useful cue, existing methods face difficulties in ellipse fitting due to unreliable iris contour detection. These methods may fail frequently especially with low resolution eye images. In this paper, we propose a synthetic iris appearance fitting (SIAF) method that is model-driven to compute 3D gaze direction from iris shape. Instead of fitting an ellipse based on exactly detected iris contour, our method first synthesizes a set of physically possible iris appearances and then optimizes inside this synthetic space to find the best solution to explain the captured eye image. In this way, the solution is highly constrained and guaranteed to be physically feasible. In addition, the proposed advanced image analysis techniques also help the SIAF method be robust to the unreliable iris contour detection. Furthermore, with multiple eye images, we propose a SIAF-joint method that can further reduce the gaze error by half, and it also resolves the binary ambiguity which is inevitable in conventional methods based on simple ellipse fitting. <s> BIB010 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> Human Computer Interaction (HCI) has become an important focus of both computer science researches and industrial applications. And, on-screen gaze estimation is one of the hottest topics in this rapidly growing field. Eye-gaze direction estimation is a sub-research area of on-screen gaze estimation and the number of studies that focused on the estimation of on-screen gaze direction is limited. Due to this, various appearance-based video-oculography methods are investigated in this work. Firstly, a new dataset is created via user images taken from daylight censored cameras located at computer screen. Then, Local Binary Pattern Histogram (LBPH), which is used in this work for the first time to obtain on-screen gaze direction information, and Principal Component Analysis (PCA) methods are employed to extract image features. And, parameter optimized Support Vector Machine (SVM), Artificial Neural Networks (ANNs) and k-Nearest Neighbor (k-NN) learning methods are adopted in order to estimate on-screen gaze direction. Finally, these methods' abilities to correctly estimate the on-screen gaze direction are compared using the resulting classification accuracies of applied methods and previous works. The best classification accuracy of 96.67% is obtained when using LBPH and SVM method pair which is better than previous works. The results also show that appearance based methods are pretty applicable for estimating on-screen gaze direction. <s> BIB011 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> Human can quickly and effortlessly focus on a few most interesting points in an image. Different observers tend to have the same fixations towards the same scene. In order to predict observer's fixations, eye gaze information can be used to reveal human attention and interest. This paper presents a real-time eye gaze tracking system. Haar cascade classifier is used to calculate the position of eye gaze based on the rectangular features of human eye. Then this position is adopted to match the space coordinates of screen representing where an observer is looking. The experimental results from different kinds of scenes validate the effectiveness of our system. <s> BIB012 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> In order to enhance the practicability and accuracy of the eye-gaze tracking system, a new type low pixel eye feature point location method is adopted. This method can accurately extract the eye-gaze features, namely iris centre point and canthus points when the image pickup requirements are low. The eye-gaze tracking method based on particle swarm optimization (PSO) BP neural network is raised, to capture pictures of eyes under the same environment, and a regression model where the connection weights and threshold values are optimized by PSO algorithm is built via BP network. This method is free of the inherent defects of BP network. This method requires only a common camera and normal illumination intensity rather than high-standard hardware, which greatly cuts the restrictive requirements for the system hardware and thus enhances the system practicability. The experiment results show that PSO-BP model is of higher robustness and accuracy than BP model, and is of higher recognition rate and can effectively enhances the eye-gaze tracking accuracy. <s> BIB013 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> Estimation eye gaze direction is useful in various human-computer interaction tasks. Knowledge of gaze direction can give valuable information regarding users point of attention. Certain patterns of eye movements known as eye accessing cues are reported to be related to the cognitive processes in the human brain. We propose a real-time framework for the classification of eye gaze direction and estimation of eye accessing cues. In the first stage, the algorithm detects faces using a modified version of the Viola-Jones algorithm. A rough eye region is obtained using geometric relations and facial landmarks. The eye region obtained is used in the subsequent stage to classify the eye gaze direction. A convolutional neural network is employed in this work for the classification of eye gaze direction. The proposed algorithm was tested on Eye Chimera database and found to outperform state of the art methods. The computational complexity of the algorithm is very less in the testing phase. The algorithm achieved an average frame rate of 24 fps in the desktop environment. <s> BIB014 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Appearance based methods <s> From scientific research to commercial applications, eye tracking is an important tool across many domains. Despite its range of applications, eye tracking has yet to become a pervasive technology. We believe that we can put the power of eye tracking in everyone's palm by building eye tracking software that works on commodity hardware such as mobile phones and tablets, without the need for additional sensors or devices. We tackle this problem by introducing GazeCapture, the first large-scale dataset for eye tracking, containing data from over 1450 people consisting of almost 2:5M frames. Using GazeCapture, we train iTracker, a convolutional neural network for eye tracking, which achieves a significant reduction in error over previous approaches while running in real time (10–15fps) on a modern mobile device. Our model achieves a prediction error of 1.71cm and 2.53cm without calibration on mobile phones and tablets respectively. With calibration, this is reduced to 1.34cm and 2.12cm. Further, we demonstrate that the features learned by iTracker generalize well to other datasets, achieving state-of-the-art results. The code, data, and models are available at http://gazecapture.csail.mit.edu. <s> BIB015
In appearance based methods the information from the eye region is represented using a model trained with a set of BIB001 . Four light sources are used at four corners of the monitor screen (only one is shown here. V1 and R1 are the virtual projection and corneal reflection of L1, Rc is the reflection and the virtual projection of the LED fixed at the camera's optical axis. V1, R1 and Rc are projected to the image plane as Uv1, Ur1, and Urc. features extracted from eye images. In Bacivarov BIB002 , a statistical model is used to represent shape and texture variations and trained using images of the eye region annotated with landmark points (Fig.6) . The shape vector is the concatenated coordinates of all landmark points, stated as where L is the number of landmark points. The shape model is obtained by applying Principal Component Analysis (PCA) on the set of aligned shapes (equations derived from BIB002 ): s = S +φ s b s , where S is the mean shape vector, and N s is the number of shape observations; φ s is the matrix having the eigenvectors as its columns; b s is the set of shape parameters. Similarly, the texture vector defined for each training image is: t=(t 1 ,t 2 ,…,t p ) T (p : number of texture samples). The texture model is derived by means of PCA on the texture vectors as (N t: : number of texture observations, , T : mean texture vector) t=T +φ t b t (16) The sets of shape and texture parameters (b t ) describe the appearance variability of the model: (W s is the vector of weights). This is the statistical model that an Active Appearance Model (AAM) algorithm uses to best fit the model to a new eye image. An active appearance based method for retrieving eye gaze from low resolution videos is presented in BIB007 . Global and local appearance models are trained and fitted for the whole face as well for capturing variance of the face and eye regions. For classifying the eye gaze into six directions, two different approaches are adopted. Gaussian Mixture Models (GMMs) are trained for large changes in gaze angles and for small gaze changes a Histograms of Oriented Gradients based method are tested. A method for 3D gaze tracking without use of active illumination is proposed in BIB010 . In this a synthetic iris appearance fitting method is introduced that computes the 3D gaze direction from iris shape. The method synthesizes a set of iris appearances and then fits the best solution to the captured eye image. This is claimed to remove unreliable iris contour detection problems arising in simple ellipse fitting and requirement of high resolution images by other methods. Once the iris contour is accurately estimated, a 3D eyeball model is used to estimate gaze from the captured eye image using the iris center/shape information. Several appearance based methods report use of local features with Support Vector Machine (SVM)s for classification of gaze direction. These include Tang BIB004 in which an AAM is used to locate the eye region using 36 feature points that represent the contour of eyes, iris size, iris location, and position of pupils. Gaze direction is estimated from 2D coordinates of feature points and is classified using an SVM. In Wang BIB003 Local-Binary-Pattern (LBP) is used to calculate the texture features and a dual camera system is used to detect the space coordinates of the eyes. These two sets of information are fed into an SVM to classify the gaze direction under natural head movement. A novel method based on Local Binary Pattern Histogram (LBPH), is used in BIB011 . LBPH and PCA are used to extract eye appearance features and several classification methods based on SVM, neural networks and k-Nearest Neighbor (k-NN)s are tested for accuracy on a collected dataset for gaze estimation. The LBPH with SVM yields best accuracy. Chen & Liu BIB008 reports the use of a special kind of discriminatory Haar features and efficient SVMs (eSVMs) for implementing a computationally efficient gaze tracking method. Haar cascade is also used in BIB012 for real time gaze tracking. Rectangular features of the eye region are calculated to extract eye and pupil regions in an image which are mapped with the gaze coordinates on screen. Neural network based approaches are used in  for headpose tolerant gaze tracking. Training data comprised of cropped eye images of a user gazing at a given point on a computer screen and corresponding coordinates of that point. Neural networks are used along with a skin color model to detect the face and eye regions in BIB005 . An improved artificial neural network optimized using the Particle Swarm Optimization approach is used for fast, high accurate and robust gaze estimation with low resolution eye images in BIB013 . Some methods include use of 3D face models as in Lai BIB006 in which head pose free gaze estimation is implemented using a such a face model with head and eye coordinate systems. Both eye appearance and head pose are considered as components of a high dimensional head pose and eye appearance (HPEA) space. A Neighbourhood Approximation Forests (NAF) approach is used to model the neighbour structure of the HPEA space followed by Adaptive Linear Regression to estimate gaze direction. Other approaches report use of a deformable model BIB008 and Genetic algorithms  . In recent times, deep learning (DL) and convolutional neural network (CNN) based methods have been proposed for gaze estimation. In BIB014 , a three stage CNN model is used to classify seven gaze directions from images taken with low cost webcams without need for calibration. Gaze tracking for a near eye display robust to illumination, skin and eye color variations and occlusion is implemented using CNNs in  . The CNN is used to learn the mapping from eye images to gaze position and comprises of two convolutional layers and two pooling with a fully connected layer at the end. In BIB015 a smartphone based app is created to collect eye images from 1450 participants which is used to train a CNN based gaze tracker that can run in real time and without calibration. The dataset comprises of images with widely varying backgrounds, lighting and head motion and the network is trained with crops of both eyes and the face region. In BIB009 an extensive eye gaze dataset is built and a multi-modal CNN based method is tested. The dataset contains more than 200,000 images with variable illumination levels and eye appearances. The CNN uses a convolutional layer followed by a max-pooling layer and second convolution layer followed by a max-pooling layer, finally with a fully connected layer. The CNN learns the mapping between input parameters, i.e., 2D head angle, eye image and gaze angle (output). 