A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> We introduce a new method for computing the 3D position of an eye and its gaze direction from a single camera and at least two near infra-red light sources. The method is based on the theory of spherical optical surfaces and uses the Gullstrand model of the eye to estimate the positions of the center of the cornea and the center of the pupil in 3D. The direction of gaze can then be computed from the vector connecting these two points. The point of regard can also be computed from the intersection of the direction of gaze with an object in the scene. We have simulated this model using ray traced images of the eye, and obtained very promising results. The major contribution of this new technique over current eye tracking technology is that the system does not require to be calibrated with the user before each user session, and it allows for free head motion. <s> BIB001 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Monocular gaze estimation is usually performed by locating the pupils, and the inner and outer eye corners in the image of the driver’s head. Of these feature points, the eye corners are just as important, and perhaps harder to detect, than the pupils. The eye corners are usually found using local feature detectors and trackers. In this paper, we describe a monocular driver gaze tracking system which uses a global head model, specifically an Active Appearance Model (AAM), to track the whole head. From the AAM, the eye corners, eye region, and head pose are robustly extracted and then used to estimate the gaze. <s> BIB002 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> This paper presents a review of eye gaze tracking technology and focuses on recent advancements that might facilitate its use in general computer applications. Early eye gaze tracking devices were appropriate for scientific exploration in controlled environments. Although it has been thought for long that they have the potential to become important computer input devices as well, the technology still lacks important usability requirements that hinders its applicability. We present a detailed description of the pupil-corneal reflection technique due to its claimed usability advantages, and show that this method is still not quite appropriate for general interactive applications. Finally, we present several recent techniques for remote eye gaze tracking with improved usability. These new solutions simplify or eliminate the calibration procedure and allow free head motion. <s> BIB003 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Interactive applications that make use of eye tracking have traditionally been based on command-and-control. Applications that make more subtle use of eye gaze have recently become increasingly popular in the domain of attentive interfaces that adapt their behaviour based on the visual attention of the user. We provide a review of the main systems and application domains where this genre of interfaces has been used. <s> BIB004 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Of all senses, it is visual perception that is predominantly deluded in Virtual Realities. Yet, the eyes of the observer, despite the fact that they are the fastest perceivable moving body part, have gotten relatively little attention as an interaction modality. A solid integration of gaze, however, provides great opportunities for implicit and explicit human-computer interaction. We present our work on integrating a lightweight head-mounted eye tracking system in a CAVE-like Virtual Reality Set-Up and provide promising data from a user study on the achieved accuracy and latency. <s> BIB005 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> We present a low-cost wearable eye tracker built from off-the-shelf components. Based on the open source openEyes project (the only other similar effort that we are aware of), our eye tracker operates in the visible spectrum and variable lighting conditions. The novelty of our approach rests in automatically switching between tracking the pupil/iris boundary in bright light to tracking the iris/sclera boundary (limbus) in dim light. Additional improvements include a semi-automatic procedure for calibrating the eye and scene cameras, as well as an automatic procedure for initializing the location of the pupil in the first image frame. The system is accurate to two degrees visual angle in both indoor and outdoor environments. <s> BIB006 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Homography normalization is presented as a novel gaze estimation method for uncalibrated setups. The method applies when head movements are present but without any requirements to camera calibration or geometric calibration. The method is geometrically and empirically demonstrated to be robust to head pose changes and despite being less constrained than cross-ratio methods, it consistently performs favorably by several degrees on both simulated data and data from physical setups. The physical setups include the use of off-the-shelf web cameras with infrared light (night vision) and standard cameras with and without infrared light. The benefits of homography normalization and uncalibrated setups in general are also demonstrated through obtaining gaze estimates (in the visible spectrum) using only the screen reflections on the cornea. <s> BIB007 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Handheld mobile devices that have a touch screen are widely used but are awkward to use with one hand. To solve this problem, we propose MobiGaze, which is a user interface that uses one's gaze (gaze interface) to operate a handheld mobile device. By using stereo cameras, the user's line of sight is detected in 3D, enabling the user to interact with a mobile device by means of his/her gaze. We have constructed a prototype system of MobiGaze that consists of two cameras with IR-LED, a Windows-based notebook PC, and iPod touch. Moreover, we have developed several applications for MobiGaze. <s> BIB008 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Despite active research and significant progress in the last 30 years, eye detection and tracking remains challenging due to the individuality of eyes, occlusion, variability in scale, location, and light conditions. Data on eye location and details of eye movements have numerous applications and are essential in face detection, biometric identification, and particular human-computer interaction tasks. This paper reviews current progress and state of the art in video-based eye detection and tracking in order to identify promising techniques as well as issues to be further addressed. We present a detailed review of recent eye models and techniques for eye detection and tracking. We also survey methods for gaze estimation and compare them based on their geometric properties and reported accuracies. This review shows that, despite their apparent simplicity, the development of a general eye detection technique involves addressing many challenges, requires further theoretical developments, and is consequently of interest to many other domains problems in computer vision and beyond. <s> BIB009 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Robust and accurate pupil detection is a prerequisite for gaze detection. Hence, we propose a new eye/pupil detection method for gaze detection on a large display. The novelty of our research can be summarized by the following four points. First, in order to overcome the performance limitations of conventional methods of eye detection, such as adaptive boosting (Adaboost) and continuously adaptive mean shift (CAMShift) algorithms, we propose adaptive selection of the Adaboost and CAMShift methods. Second, this adaptive selection is based on two parameters: pixel differences in successive images and matching values determined by CAMShift. Third, a support vector machine (SVM)-based classifier is used with these two parameters as the input, which improves the eye detection performance. Fourth, the center of the pupil within the detected eye region is accurately located by means of circular edge detection, binarization and calculation of the geometric center. The experimental results show that the proposed method can detect the center of the pupil at a speed of approximately 19.4 frames/s with an RMS error of approximately 5.75 pixels, which is superior to the performance of conventional detection methods. <s> BIB010 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Despite the widespread use of mobile phones and tablets, hand-held portable devices have only recently been identified as a promising platform for gaze-aware applications. Estimating gaze on portable devices is challenging given their limited computational resources, low quality integrated front-facing RGB cameras, and small screens to which gaze is mapped. In this paper we present EyeTab, a model-based approach for binocular gaze estimation that runs entirely on an unmodified tablet. EyeTab builds on set of established image processing and computer vision algorithms and adapts them for robust and near-realtime gaze estimation. A technical prototype evaluation with eight participants in a normal indoors office setting shows that EyeTab achieves an average gaze estimation accuracy of 6.88° of visual angle at 12 frames per second. <s> BIB011 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> In this work, a new head mounted eye tracking system is presented. Based on computer vision techniques, the system integrates eye images and head movement, in real time, performing a robust gaze point tracking. Nystagmus movements due to vestibulo-ocular reflex are monitored and integrated. The system proposed here is a strongly improved version of a previous platform called HATCAM, which was robust against changes of illumination conditions. The new version, called HAT-Move, is equipped with accurate inertial motion unit to detect the head movement enabling eye gaze even in dynamical conditions. HAT-Move performance is investigated in a group of healthy subjects in both static and dynamic conditions, i.e. when head is kept still or free to move. Evaluation was performed in terms of amplitude of the angular error between the real coordinates of the fixed points and those computed by the system in two experimental setups, specifically, in laboratory settings and in a 3D virtual reality (VR) scenario. The achieved results showed that HAT-Move is able to achieve eye gaze angular error of about 1 degree along both horizontal and vertical directions <s> BIB012 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Over the last several years, there has been much research and investigation on finding new ways to interact with the smart systems that currently form an integral part of our lives. One of the most widely researched fields in human?computer interaction (HCI) has been the use of human eye gaze as an input modality to control and command intelligent systems. For example, gaze-based schemes for hands-free input to computers for text entry/scrolling/pointing was proposed as early as in 1989 for disabled persons [1]. In the field of commercial applications, gaze-based interactions have brought immersive experiences in the world of virtual gaming and multimedia entertainment [2]. Eye gaze is also a significant feature for detecting the attention and intent of an individual. For example, gaze tracking could be implemented in a car to detect driver consciousness or in a smartphone to switch operations by sensing user attentiveness. <s> BIB013 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> There is a need to standardize the performance of eye gaze estimation (EGE) methods in various platforms for human computer interaction (HCI). Because of lack of consistent schemes or protocols for summative evaluation of EGE systems, performance results in this field can neither be compared nor reproduced with any consistency. In contemporary literature, gaze tracking accuracy is measured under non-identical sets of conditions, with variable metrics and most results do not report the impact of system meta-parameters that significantly affect tracking performances. In this work, the diverse nature of these research outcomes and system parameters which affect gaze tracking in different platforms is investigated and their error contributions are estimated quantitatively. Then the concept and development of a performance evaluation framework is proposed- that can define design criteria and benchmark quality measures for the eye gaze research community. <s> BIB014 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> I. INTRODUCTION <s> Several generic CE use cases and corresponding techniques for eye gaze estimation (EGE) are reviewed. The optimal approaches for each use case are determined from a review of recent literature. In addition, the most probable error sources for EGE are determined and the impact of these error sources is quantified. A discussion and analysis of the research outcome is given and future work outlined. <s> BIB015
DVANCES in eye gaze tracking technology over the past few decades have led to the development of promising gaze estimation techniques and applications for human computer interaction. Historically, research on gaze tracking dates back to the early 1900s, starting with invasive eye tracking techniques. These included electro-occulography using pairs of electrodes placed around the eyes or the scleral search methods that include coils embedded into a contact lens adhering to the eyes. The first video based eye tracking study was made on pilots operating airplane controls in the 1940s  . Research on head-mounted eye trackers advanced in the 1960s and gaze tracking developed further in the 1970s with focus on improving accuracy and reducing the constraints on users. With increasing computing power in devices, real time operation of eye trackers became possible during the 1980s. However till this time, owing to limited availability of computers, eye tracking was mainly limited to psychological and cognitive studies and medical research. The application focus towards general purpose human computer interaction was sparse. This changed in the 1990s as eye gaze found applications in computer input and control [2] . Post 2000, rapid advancements in computing speed, digital video processing and low cost hardware brought gaze tracking equipment closer to users, with applications in gaming, virtual reality and web-advertisements BIB003 . Eye gaze information is used in a variety of user platforms. The main use cases may be broadly classified into (i) desktop computers BIB001 - BIB007 , (ii) TV panels BIB010 [8], (iii) head mounted BIB012 - BIB006 (iv) automotive setups BIB002 -  (v) handheld devices BIB011 BIB008 . Applications based on desktop platforms involve using eye gaze for computer communication and text entry, computer control and entering gaze based passwords BIB004 . Remote eye tracking has recently been used on TV panels to achieve gaze controlled functions, for example selecting and navigating menus and switching channels. Head-mounted gaze tracking setups usually comprise of two or more cameras mounted on a support framework worn by the user. Such systems have been extensively employed in user attention and cognitive studies, psychoanalysis, occulo-motor measurements [2] , virtual and augmented reality applications BIB005  . Real time gaze and eye state tracking on automotive platforms is used in driver support systems to evaluate driver vigilance and drowsiness levels. These use eye tracking setups mounted on a car's dashboard along with computing hardware running machine vision algorithms. In handheld devices such as smartphones or tablets, the front camera is used to track user gaze to activate functions such as locking/unlocking phones, interactive displays, dimming backlights or suspending sensors BIB011 BIB013 . Within each of these use cases there exists a wide range of system configurations, operating conditions and varying quality of imaging and optical components. Furthermore, the variations in eye-movement and biological aspects of individuals lead to challenges in achieving consistent and repeatable performance from gaze tracking methods. Thus, despite several decades of development in eye gaze research, performance evaluation and comparison of different gaze estimation techniques across different platforms is a still a difficult task BIB014 BIB015 . In order to provide insight into the current status of eye gaze research and outcomes, this paper presents a detailed literature review and analysis that considers algorithms, system A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms Anuradha Kar, Student Member, IEEE, Peter Corcoran Fellow, IEEE A configuration, user conditions and performance issues for existing gaze tracking systems. Specifically, use-cases based on five different eye gaze platforms are considered. The aim of this work is to gain a realistic overview of the diversity currently existing in this field and to identify the factors that affect the practical usability of gaze tracking systems. Further, this review highlights the need for developing standardized measurement protocols to enable evaluation and comparison of the performance and operational characteristics of different gaze tracking systems. In this paper, first the diversity and standardization issues in different aspects of eye gaze research are discussed and then the idea of a performance evaluation framework is proposed. This framework includes several planned and ongoing experiments that are aimed at practical evaluation of any gaze tracker. Our goal is to encourage further discussion and additional contributions from researchers in this field. There have been detailed review works on eye gaze made in the last few years such as BIB003 , BIB009 -  which discussed about recent developments in gaze tracking methods, comparing different estimation techniques, setups, applications and challenges involved in using gaze as an input modality. Hansen BIB009 provides an in-depth review on different eye models, eye detection techniques and models for gaze estimation, along with a summary of gaze applications. It also discusses inaccuracies in gaze tracking arising from the eye model components, jitter and refraction due to user wearing glasses. However, our work differs on several grounds from these reviews. Firstly, our review is specifically aimed towards highlighting the issues affecting realistic performance evaluation of gaze tracking systems, such as ambiguous accuracy metrics and un-accounted error sources. Secondly, we do a detailed classification of four different eye gaze research platforms. Extensive literature resources are collected and analyzed for each platform with the aim to understand the factors that affect the performance of a gaze based system in each of these. Thirdly, we present our survey in a statistical format that shows the lack of standardization in eye gaze research as a quantitative observation. Also our survey includes research works published until 2017 to make it exhaustive and up-to-date. Finally, our review not only provides an overview of the current status of eye gaze research but also forms the foundation of a performance evaluation framework for eye gaze systems which is proposed by us in Section VI of this work and is currently under development. The paper is organized as follows: Section II presents a brief overview on eye movements, gaze tracking systems and accuracy measures used in contemporary gaze research. In Sections III and IV, several gaze tracking algorithms are categorized and key research works on the implementation of gaze tracking in five different user platforms are reviewed. In Section V, the factors limiting practical performance of gaze tracking in different user platforms are analyzed and issues with diversity in gaze accuracy metrics are discussed. The background and concept of a methodological framework for practical evaluation of eye gaze systems is presented in Section VI. We note here that the scope of this review excludes gaze tracking for clinical and neurological directions, retinal imaging and studies on children and patients.