A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> We introduce a new method for computing the 3D position of an eye and its gaze direction from a single camera and at least two near infra-red light sources. The method is based on the theory of spherical optical surfaces and uses the Gullstrand model of the eye to estimate the positions of the center of the cornea and the center of the pupil in 3D. The direction of gaze can then be computed from the vector connecting these two points. The point of regard can also be computed from the intersection of the direction of gaze with an object in the scene. We have simulated this model using ray traced images of the eye, and obtained very promising results. The major contribution of this new technique over current eye tracking technology is that the system does not require to be calibrated with the user before each user session, and it allows for free head motion. <s> BIB001 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> In the eye gaze tracking problem, the goal is to determine where on a monitor screen a computer user is looking, ie., the gaze point. Existing systems generally have one of two limitations: either the head must remain fixed in front of a stationary camera, or, to allow for head motion, the user must wear an obstructive device. We introduce a 3D eye tracking system where the head motion is allowed without the need for markers or worn devices. We use a pair of stereo systems: a wide angle stereo system detects the face and steers an active narrow FOV stereo system to track the eye at high resolution. For high resolution tracking, the eye is modeled in 3D, including the corneal ball, pupil and fovea. We discuss the calibration of the stereo systems, the eye model, eye detection and tracking, and we close with an evaluation of the accuracy of the estimated gaze point on the monitor. <s> BIB002 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> Human eye gaze is a strong candidate to create a new application area based on human-computer interaction. To implement a really practical gaze-based interaction system, gaze detection must be realized without placing any restriction on the user's behavior or comfort. This paper describes a gaze tracking system that offers freehead, simple personal calibration. It does not require the user wear anything on her head, and she can move her head freely. Personal calibration takes only a very short time; the user is asked to look at two markers on the screen. An experiment shows that the accuracy of the implemented system is about 1.0 degrees (view angle). <s> BIB003 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> A novel approach to three-dimensional (3-D) gaze tracking using 3-D computer vision techniques is proposed in this paper. This method employs multiple cameras and multiple point light sources to estimate the optical axis of user's eye without using any user-dependent parameters. Thus, it renders the inconvenient system calibration process which may produce possible calibration errors unnecessary. A real-time 3-D gaze tracking system has been developed which can provide 30 gaze measurements per second. Moreover, a simple and accurate calibration method is proposed to calibrate the gaze tracking system. Before using the system, each user only has to stare at a target point for a few (2-3) seconds so that the constant angle between the 3-D line of sight and the optical axis can be estimated. The test results of six subjects showed that the gaze tracking system is very promising achieving an average estimation error of under 1/spl deg/. <s> BIB004 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> Many eye-tracking systems either require the user to keep their head still or involve cameras or other equipment mounted on the user's head. While acceptable for research applications, these limitations make the systems unsatisfactory for prolonged use in interactive applications. Since the goal of our work is to use eye trackers for improved visual communication through gaze guidance [1,2] and for Augmentative and Alternative Communication (AAC) [3], we are interested in less invasive eye tracking techniques. <s> BIB005 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> This paper presents a general theory for the remote estimation of the point-of-gaze (POG) from the coordinates of the centers of the pupil and corneal reflections. Corneal reflections are produced by light sources that illuminate the eye and the centers of the pupil and corneal reflections are estimated in video images from one or more cameras. The general theory covers the full range of possible system configurations. Using one camera and one light source, the POG can be estimated only if the head is completely stationary. Using one camera and multiple light sources, the POG can be estimated with free head movements, following the completion of a multiple-point calibration procedure. When multiple cameras and multiple light sources are used, the POG can be estimated following a simple one-point calibration procedure. Experimental and simulation results suggest that the main sources of gaze estimation errors are the discrepancy between the shape of real corneas and the spherical corneal shape assumed in the general theory, and the noise in the estimation of the centers of the pupil and corneal reflections. A detailed example of a system that uses the general theory to estimate the POG on a computer screen is presented. <s> BIB006 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> Eye-gaze as a form of human machine interface holds great promise for improving the way we interact with machines. Eye-gaze tracking devices that are non-contact, non-restrictive, accurate and easy to use will increase the appeal for including eye-gaze information in future applications. The system we have developed and which we describe in this paper achieves these goals using a single high resolution camera with a fixed field of view. The single camera system has no moving parts which results in rapid reacquisition of the eye after loss of tracking. Free head motion is achieved using multiple glints and 3D modeling techniques. Accuracies of under 1° of visual angle are achieved over a field of view of 14x12x20 cm and over various hardware configurations, camera resolutions and frame rates. <s> BIB007 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> Most available remote eye gaze trackers have two characteristics that hinder them being widely used as the important computer input devices for human computer interaction. First, they have to be calibrated for each user individually; second, they have low tolerance for head movement and require the users to hold their heads unnaturally still. In this paper, by exploiting the eye anatomy, we propose two novel solutions to allow natural head movement and minimize the calibration procedure to only one time for a new individual. The first technique is proposed to estimate the 3D eye gaze directly. In this technique, the cornea of the eyeball is modeled as a convex mirror. Via the properties of convex mirror, a simple method is proposed to estimate the 3D optic axis of the eye. The visual axis, which is the true 3D gaze direction of the user, can be determined subsequently after knowing the angle deviation between the visual axis and optic axis by a simple calibration procedure. Therefore, the gaze point on an object in the scene can be obtained by simply intersecting the estimated 3D gaze direction with the object. Different from the first technique, our second technique does not need to estimate the 3D eye gaze directly, and the gaze point on an object is estimated from a gaze mapping function implicitly. In addition, a dynamic computational head compensation model is developed to automatically update the gaze mapping function whenever the head moves. Hence, the eye gaze can be estimated under natural head movement. Furthermore, it minimizes the calibration procedure to only one time for a new individual. The advantage of the proposed techniques over the current state of the art eye gaze trackers is that it can estimate the eye gaze of the user accurately under natural head movement, without need to perform the gaze calibration every time before using it. Our proposed methods will improve the usability of the eye gaze tracking technology, and we believe that it represents an important step for the eye tracker to be accepted as a natural computer input device. <s> BIB008 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> This paper presents a calibration-free method for estimating the point of gaze (POG) on a display by using two pairs of stereo cameras. By using one pair of cameras and two light sources, the optical axis of the eye and the position of the center of the cornea can be estimated. This estimation is carried out by using a spherical model of the cornea. One pair of cameras is used for the estimation of the optical axis of the left eye, and the other pair is used for the estimation of the optical axis of the right eye. The point of intersection of optical axis with the display is termed the point of the optical axis (POA). The POG is approximately estimated as the midpoint of the line joining POAs of both the eyes with the display. We have developed a prototype system based on this method and demonstrated that the midpoint of POAs was closer to the fiducial point that the user gazed at than each POA. <s> BIB009 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> A novel gaze estimation method based on a novel aspherical model of the cornea is proposed in this paper. The model is a surface of revolution about the optical axis of the eye. The calculation method is explained on the basis of the model. A prototype system for estimating the point of gaze (POG) has been developed using this method. The proposed method has been found to be more accurate than the gaze estimation method based on a spherical model of the cornea. <s> BIB010 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> A novel general method to extend the tracking range of user-calibration-free remote eye-gaze tracking (REGT) systems that are based on the analysis of stereo-images from multiple cameras is presented. The method consists of two distinct phases. In the brief initial phase, estimates of the center of the pupil and corneal reflections in pairs of stereo-images are used to estimate automatically a set of subject-specific eye parameters. In the second phase, these subject — specific eye parameters are used with estimates of the center of the pupil and corneal reflections in images from any one of the systems' cameras to compute the Point-of-Gaze (PoG). Experiments with a system that includes two cameras show that the tracking range for horizontal gaze directions can be extended from ±23.2° when the two cameras are used as a stereo pair to ±35.5° when the two cameras are used independently to estimate the POG. <s> BIB011 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> Glint features have important roles in gaze-tracking systems. However, when the operation range of a gaze-tracking system is enlarged, the performance of glint-feature-based (GFB) approaches will be degraded mainly due to the curvature variation problem at around the edge of the cornea. Although the pupil contour feature may provide complementary information to help estimating the eye gaze, existing methods do not properly handle the cornea refraction problem, leading to inaccurate results. This paper describes a contour-feature-based (CFB) 3-D gaze-tracking method that is compatible to cornea refraction. We also show that both the GFB and CFB approaches can be formulated in a unified framework and, thus, they can be easily integrated. Furthermore, it is shown that the proposed CFB method and the GFB method should be integrated because the two methods provide complementary information that helps to leverage the strength of both features, providing robustness and flexibility to the system. Computer simulations and real experiments show the effectiveness of the proposed approach for gaze tracking. <s> BIB012 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> Traditional gaze tracking systems rely on explicit infrared lights and high resolution cameras to achieve high performance and robustness. These systems, however, require complex setup and thus are restricted in lab research and hard to apply in practice. In this paper, we propose to perform gaze tracking with a consumer level depth sensor (Kinect). Leveraging on Kinect's capability to obtain 3D coordinates, we propose an efficient model-based gaze tracking system. We first build a unified 3D eye model to relate gaze directions and eye features (pupil center, eyeball center, cornea center) through subject-dependent eye parameters. A personal calibration framework is further proposed to estimate the subject-dependent eye parameters. Finally we can perform real time gaze tracking given the 3D coordinates of eye features from Kinect and the subject-dependent eye parameters from personal calibration procedure. Experimental results with 6 subjects prove the effectiveness of the proposed 3D eye model and the personal calibration framework. Furthermore, the gaze tracking system is able to work in real time (20 fps) and with low resolution eye images. <s> BIB013 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> B. 3D model based methods <s> In this paper, we address the 3D eye gaze estimation problem using a low-cost, simple-setup, and non-intrusive consumer depth sensor (Kinect sensor). We present an effective and accurate method based on 3D eye model to estimate the point of gaze of a subject with the tolerance of free head movement. To determine the parameters involved in the proposed eye model, we propose i) an improved convolution-based means of gradients iris center localization method to accurately and efficiently locate the iris center in 3D space; ii) a geometric constraints-based method to estimate the eyeball center under the constraints that all the iris center points are distributed on a sphere originated from the eyeball center and the sizes of two eyeballs of a subject are identical; iii) an effective Kappa angle calculation method based on the fact that the visual axes of both eyes intersect at a same point with the screen plane. The final point of gaze is calculated by using the estimated eye model parameters. We experimentally evaluate our gaze estimation method on five subjects. The experimental results show the good performance of the proposed method with an average estimation accuracy of 3.78°, which outperforms several state-of-the-arts. <s> BIB014
These methods use a geometrical model of the human eye to estimate the center of the cornea, optical and visual axes of the eye (Fig. 4a) and estimate the gaze coordinates as points of intersection where the visual axes meets the scene. 3D model based methods can be categorized on the basis of whether they use single or multiple cameras and type of user calibration required. 3D model based methods using single camera have been reported by Martinez BIB005 , Guestrin BIB006 and Hennessey BIB007 . Single camera systems have simple system geometry, no moving parts and fast re-acquisition capabilities. For 3D gaze estimation in Martinez BIB005 a single camera and LED are used to achieve an accuracy of 0.5 degrees with user calibration. Guestrin presents a mathematical model to reconstruct the optical and visual axes of the user's eyes from the centres of the pupil and glint in the captured video frames and configuration of a remote gaze tracking system. The model considers single and multiple cameras and light sources in estimation of the point of gaze. It then demonstrates the gaze tracking performance of a system implemented using two NIR light sources and one camera using the model .Their method achieves an accuracy of around 0.9 degrees. The system proposed by Hennessey includes a single camera and multiple LEDs to achieve 3D gaze tracking with free head motion. Multi camera methods achieve high accuracy and robustness against head movement but require elaborate system calibration procedures including calibration of cameras for 3D measurements, estimating positioning of LEDs and determining the geometric properties of the monitors and their relation with the cameras. Some key works using two or more cameras include Shih BIB012 , Ohno BIB003 , Beymer BIB002 and Zhu BIB008 . Ohno describes 3D gaze tracking allowing free head motion using simple two point calibration and a two camera system comprising of an eye positioning unit and a gaze detection unit. The eye positioning unit uses narrow field stereo cameras and controls direction of the gaze positioning unit to achieve head motion independent tracking. A head pose free gaze tracking system is also implemented by Beymer with a wide angle stereo system for eye positioning and narrow angle stereo system for gaze detection. Pan and tilt directions of the narrow angle camera are controlled using rotating mirrors with galvo-motors. Shih and Liu BIB004 used a simplified eye model by Le Grand with two cameras and two LEDs to estimate the optical axis of the eye through solving linear equations. It uses single point calibration. The method in Zhu uses a gaze mapping function along with a dynamic head compensation model to update the gaze mapping function whenever the head moves to achieve tracking under natural head movement. It uses a 2 camera system with one time user calibration. With respect to the model of human eye, BIB006 provided evidence that a fully spherical corneal model will result in no impact of head movements on gaze estimation. They assumed an ellipsoidal model of the cornea and reported that gaze estimation errors increase with corneal asphericity and this also results in sensitivity of gaze estimation to head movements. An aspherical model of the cornea BIB010 is shown in Fig 4b and it is a surface of revolution about the optical axis of the eye. Use of this model showed to result in better accuracy, especially near the display corners as compared to a traditional 3D model based method. Calibration free gaze estimation techniques have been proposed by Nagamatsu BIB009 , Model BIB011 and Morimoto BIB001 . In BIB009 , a calibration-free method is proposed using two pairs of stereo cameras, light sources and a spherical model of the cornea. One pair of cameras and two light sources are used for each eye to estimate the eye optical axis and the position of the center of the cornea. Optical axes of both eyes are measured using a binocular 3D eye model to estimate the point of gaze, achieving an accuracy of around 2.0°. Model & Eizenman also proposed a multiple camera based system to capture stereo images of the eye with corneal reflections. From the stereo eye images, eye features, such as the center of the pupil and corneal reflections are used to estimate subjectspecific eye parameters. These parameters are then used with the eye features to estimate PoG. Their method is calibration free, has a tracking range of 3 to 5 meters and accuracy of less than 2 degrees. Morimoto proposed a method using two light sources and one camera that doesn't require user calibration for every session. It uses the Gullstrand model of the eye and ray tracing techniques to estimate the cornea and pupil centers. It achieves an accuracy of 2-4 degrees of visual angle dependent on the position of the light sources. A new class of 3D gaze tracking has recently emerged with the usage of depth sensors in several works. These sensors comprise of an RGB camera and an infra-red depth camera. Typically resolution for the RGB camera is 640 x 480 pixels, with 45 degrees vertical and 58 degrees horizontal field of view. The depth camera resolution is about 1.5 mm at 50 cm. Gaze tracking using the consumer grade depth sensor (Kinect) is proposed in BIB013 . The method uses an eye model; 3D coordinates of eye features are obtained from Kinect and eye parameters like eyeball and pupil center are derived from a user calibration process. With this, 3D gaze coordinates are tracked in real time with a simple setup. Another work BIB014 reports the use of Kinect and a simple low cost setup for 3D model based gaze estimation allowing free head motion. It derives the 3D model parameters using convolution based means of gradients iris center localization method and uses a geometric constraints-based method to estimate the eyeball center. They assume that iris center points are distributed on a sphere originated from the eyeball center and the sizes of two eyeballs of a subject are identical. Kinect data is used to obtain 3D positions of person's head pose, iris and eyeball centers. [64] also uses a Kinect sensor and a model to estimate eyeball center by making users look at a target in 3D space. Kinect is used to build a head model to determine the eyeball center, detect the pupil center and determine 3D eye gaze coordinates in conjunction with the eye model.