A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> Eye tracking is one of the key technologies for future driver assistance systems since human eyes contain much information about the driver's condition such as gaze, attention level, and fatigue level. Thus, nonintrusive methods for eye detection and tracking are important for many applications of vision-based driver-automotive interaction. One problem common to many eye tracking methods proposed so far is their sensitivity to lighting condition change. This tends to significantly limit their scope for automotive applications. In this paper we present a new realtime eye detection and tracking method that works under variable and realistic lighting conditions. By combining imaging by using IR light and appearance-based object recognition techniques, our method can robustly track eyes even when the pupils are not very bright due to significant external illumination interferences. The appearance model is incorporated in both eye detection and tracking via the use of a support vector machine and mean shift tracking. Our experimental results show the feasibility of our approach and the validity for the method is extended for drivers wearing sunglasses. <s> BIB001 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> This paper describes a real-time online prototype driver-fatigue monitor. It uses remotely located charge-coupled-device cameras equipped with active infrared illuminators to acquire video images of the driver. Various visual cues that typically characterize the level of alertness of a person are extracted in real time and systematically combined to infer the fatigue level of the driver. The visual cues employed characterize eyelid movement, gaze movement, head movement, and facial expression. A probabilistic model is developed to model human fatigue and to predict fatigue based on the visual cues obtained. The simultaneous use of multiple visual cues and their systematic combination yields a much more robust and accurate fatigue characterization than using a single visual cue. This system was validated under real-life fatigue conditions with human subjects of different ethnic backgrounds, genders, and ages; with/without glasses; and under different illumination conditions. It was found to be reasonably robust, reliable, and accurate in fatigue characterization. <s> BIB002 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> In this paper we present a non-intrusive prototype computer vision system for real-time monitoring driver's vigilance. It is based on a hardware system, for real time acquisition of driver's images using an active IR illuminator, and their software implementation for monitoring some visual behaviors that characterize a driver's level of vigilance. These are the eyelid movements and the pose face. The system has been tested with different sequences recorded on night and day driving conditions in a motorway and with different users. We show some experimental results and some conclusions about the performance of the system. <s> BIB003 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> The driver assistant system has attracted much attention as an essential component of intelligent transportation systems. One task of driver assistant system is to prevent the drivers from fatigue. For the fatigue detection it is natural that the information about eyes should be utilized. The driver fatigue can be divided into two types, one is the sleep with eyes close and another is the sleep with eyes open. Considering that the fatigue detection is related with the prior knowledge and probabilistic statistics, the dynamic Bayesian network is used as the analysis tool to perform the reasoning of fatigue. Two kinds of experiments are performed to verify the system effectiveness, one is based on the video got from the laboratory and another is based on the video got from the real driving situation. Ten persons participate in the test and the experimental result is that, in the laboratory all the fatigue events can be detected, and in the practical vehicle the detection ratio is about 85%. Experiments show that in most of situations the proposed system works and the corresponding performance is satisfying. <s> BIB004 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> As use of in-vehicle information systems (IVISs) such as cell phones, navigation systems, and satellite radios has increased, driver distraction has become an important and growing safety concern. A promising way to overcome this problem is to detect driver distraction and adapt in-vehicle systems accordingly to mitigate such distractions. To realize this strategy, this paper applied support vector machines (SVMs), which is a data mining method, to develop a real-time approach for detecting cognitive distraction using drivers' eye movements and driving performance data. Data were collected in a simulator experiment in which ten participants interacted with an IVIS while driving. The data were used to train and test both SVM and logistic regression models, and three different model characteristics were investigated: how distraction was defined, which data were input to the model, and how the input data were summarized. The results show that the SVM models were able to detect driver distraction with an average accuracy of 81.1%, outperforming more traditional logistic regression models. The best performing model (96.1% accuracy) resulted when distraction was defined using experimental conditions (i.e., IVIS drive or baseline drive), the input data were comprised of eye movement and driving measures, and these data were summarized over a 40-s window with 95% overlap of windows. These results demonstrate that eye movements and simple measures of driving performance can be used to detect driver distraction in real time. Potential applications of this paper include the design of adaptive in-vehicle systems and the evaluation of driver distraction <s> BIB005 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> This paper presents a vision-based real-time gaze zone estimator based on a driver's head orientation composed of yaw and pitch. Generally, vision-based methods are vulnerable to the wearing of eyeglasses and image variations between day and night. The proposed method is novel in the following four ways: First, the proposed method can work under both day and night conditions and is robust to facial image variation caused by eyeglasses because it only requires simple facial features and not specific features such as eyes, lip corners, and facial contours. Second, an ellipsoidal face model is proposed instead of a cylindrical face model to exactly determine a driver's yaw. Third, we propose new features-the normalized mean and the standard deviation of the horizontal edge projection histogram-to reliably and rapidly estimate a driver's pitch. Fourth, the proposed method obtains an accurate gaze zone by using a support vector machine. Experimental results from 200 000 images showed that the root mean square errors of the estimated yaw and pitch angles are below 7 under both daylight and nighttime conditions. Equivalent results were obtained for drivers with glasses or sunglasses, and 18 gaze zones were accurately estimated using the proposed gaze estimation method. <s> BIB006 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> In this paper, we propose a novel method for gaze recognition of a driver coping with rotation of a driver's face. Frontal face images and left half profile images were separately trained using the Viola-Jones (V-J) algorithm to produce classifiers that can detect faces. The right half profile can be detected by mirroring the entire image when neither a frontal face nor a left half profile was detected. As an initial step, this method was used to simultaneously detect the driver's face. Then, we applied a regressional version of linear discriminant analysis (LDAr) to the detected facial region to extract important features for classification. Finally, these features were used to classify the driver's gaze in seven directions. In the feature extraction step, LDAr tries to find features that maximize the ratio of interdistances among samples with large differences in the target value to those with small differences in the target value. Therefore, the resultant features are more fitted to regression problems than conventional feature extraction methods. In addition to LDAr, in this paper, a 2-D extension of LDAr is also developed and used as a feature extraction method for gaze recognition. The experimental results show that the proposed method achieves a good gaze recognition rate under various rotation angles of a driver's head, resulting in a reliable headlamp control performance. <s> BIB007 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> Many automated driver monitoring technologies have been proposed to enhance vehicle and road safety. Most existing solutions involve the use of specialized embedded hardware, primarily in high-end automobiles. This paper explores driver assistance methods that can be implemented on mobile devices such as a consumer smartphone, thus offering a level of safety enhancement that is more widely accessible. Specifically, the paper focuses on estimating driver gaze direction as an indicator of driver attention. Input video frames from a smartphone camera facing the driver are first processed through a coarse head pose direction. Next, the locations and scales of face parts, namely mouth, eyes, and nose, define a feature descriptor that is supplied to an SVM gaze classifier which outputs one of 8 common driver gaze directions. A key novel aspect is an in-situ approach for gathering training data that improves generalization performance across drivers, vehicles, smartphones, and capture geometry. Experimental results show that a high accuracy of gaze direction estimation is achieved for four scenarios with different drivers, vehicles, smartphones and camera locations. <s> BIB008 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> Analysis of driver's head behavior is an integral part of driver monitoring system. Driver's coarse gaze direction or gaze zone is a very important cue in understanding driver- state. Many existing gaze zone estimators are, however, limited to single camera perspectives, which are vulnerable to occlu- sions of facial features from spatially large head movements away from the frontal pose. Non-frontal glances away from the driving direction, though, are of special interest as interesting events, critical to driver safety, occur during those times. In this paper, we present a distributed camera framework for gaze zone estimation using head pose dynamics to operate robustly and continuously even during large head movements. For experimental evaluations, we collected a dataset from naturalistic on-road driving in urban streets and freeways. A human expert provided the gaze zone ground truth using all vision information including eyes and surround context. Our emphasis is to understand the efficacy of the head pose dynamic information in predicting eye-gaze-based zone ground truth. We conducted several experiments in designing the dynamic features and compared the performance against static head pose based approach. Analyses show that dynamic information significantly improves the results. <s> BIB009 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> This paper proposes a system that uses gaze direction tracking and head pose estimation to detect drowsiness of a driver. Head pose is estimated by calculating optic flow of the facial features, which are acquired with a corner detection algorithm. Analysis of the driver's head behavior leads to three moving components: nodding, shaking, and tilting. To track the gaze direction of the driver, we trace the center point of the pupil using CDF analysis and estimate the frequency of eye-movement. <s> BIB010 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> This paper comes as a response to the fact that, lately, more and more accidents are caused by people who fall asleep at the wheel. Eye tracking is one of the most important aspects in driver assistance systems since human eyes hold much in-formation regarding the driver's state, like attention level, gaze and fatigue level. The number of times the subject blinks will be taken into account for identification of the subject's drowsiness. Also the direction of where the user is looking will be estimated according to the location of the user's eye gaze. The developed algorithm was implemented on a Raspberry Pi board in order to create a portable system. The main determination of this project is to conceive an active eyetracking based system, which focuses on the drowsiness detection amongst fatigue related deficiencies in driving. <s> BIB011 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> Distracted driving is one of the main causes of vehicle collisions in the United States. Passively monitoring a driver's activities constitutes the basis of an automobile safety system that can potentially reduce the number of accidents by estimating the driver's focus of attention. This paper proposes an inexpensive vision-based system to accurately detect Eyes Off the Road (EOR). The system has three main components: 1) robust facial feature tracking; 2) head pose and gaze estimation; and 3) 3-D geometric reasoning to detect EOR. From the video stream of a camera installed on the steering wheel column, our system tracks facial features from the driver's face. Using the tracked landmarks and a 3-D face model, the system computes head pose and gaze direction. The head pose estimation algorithm is robust to nonrigid face deformations due to changes in expressions. Finally, using a 3-D geometric analysis, the system reliably detects EOR. <s> BIB012 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (1) vehicle interfaces and (2) the design of future Advanced Driver Assistance Systems. In this paper, we estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can we classify driver gaze using head and eye pose versus just using head pose? Second, are there individual-specific gaze strategies that strongly correlate with how much gaze classification improves with the addition of eye pose information? We answer these questions by evaluating data drawn from an on-road study of 40 drivers. The main insight of the paper is conveyed through the analogy of an "owl" and "lizard" which describes the degree to which the eyes and the head move when shifting gaze. When the head moves a lot ("owl"), not much classification improvement is attained by estimating eye pose on top of head pose. On the other hand, when the head stays still and only the eyes move ("lizard"), classification accuracy increases significantly from adding in eye pose. We characterize how that accuracy varies between people, gaze strategies, and gaze regions. <s> BIB013 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> With the advent of global positioning system technology, smart phones are used as portable navigation systems. Guidelines that ensure driving safety while using conventional on-board navigation systems have already been published but do not extend to portable navigation systems; therefore, this study focused on the analysis of the eye-gaze tracking of drivers interacting with portable navigation systems in an urban area. Combinations of different display sizes and positions of portable navigation systems were adopted by 20 participants in a driving simulator experiment. An expectation maximum algorithm was proposed to classify the measured eye-gaze points; furthermore, three measures of glance frequency, glance time, and total glance time as a percentage were calculated. The results indicated that the convenient display position with a small visual angle can provide a significantly shorter glance time but a significantly higher glance frequency; however, the small-size display will bring on significantly longer glance time that may result in the increasing of visual distraction for drivers. The small-size portable display received significantly lower scores for subjective evaluation of acceptability and fatigue; moreover, the small-size portable display on the conventional built-in position received significantly lower subjective evaluation scores than that of the big-size one on the upper side of the dashboard. In addition, it indicated an increased risk of rear-end collision that the proportion of time that the time-to-collision was less than 1 s was significantly shorter for the portable navigation than that of traditional on-board one. <s> BIB014 </s> A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms <s> D. Automotive <s> Monitoring driver behavior is crucial in the design of advanced driver assistance systems (ADAS) that can detect driver actions, providing necessary warnings when not attentive to driving tasks. The visual attention of a driver is an important aspect to consider, as most driving tasks require visual resources. Previous work has investigated algorithms to detect driver visual attention by tracking the head or eye movement. While tracking pupil can give an accurate direction of visual attention, estimating gaze on vehicle environment is a challenging problem due to changes in illumination, head rotations, and occlusions (e.g. hand, glasses). Instead, this paper investigates the use of the head pose as a coarse estimate of the driver visual attention. The key challenge is the non-trivial relation between head and eye movements while glancing to a target object, which depends on the driver, the underlying cognitive and visual demand, and the environment. First, we evaluate the performance of a state-of-the-art head pose detection algorithm over natural driving recordings, which are compared with ground truth estimations derived from AprilTags attached to a headband. Then, the study proposes regression models to estimate the drivers' gaze based on the head position and orientation, which are built with data from natural driving recordings. The proposed system achieves high accuracy over the horizontal direction, but moderate/low performance over the vertical direction. We compare results while our participants were driving, and when the vehicle was parked. <s> BIB015
Visual features of the face and eye regions of an automobile driver provide cues about their degree of alertness, perception and vehicle control. Knowledge about driver cognitive state helps to predict if the driver intends to change lanes or is aware about obstacles and thereby avoid fatal accidents. Gaze related cues that indicate driver attentiveness include: blink rate, temporal gaze variation, speed of eyelid movements and degree of eye openness. Several works for driver assistance systems have been reported that are based on video based gaze tracking using a variety of classifiers. SVM based gaze classification is common on automotive platforms and has been reported in Fujimura BIB001 , Park BIB006 and Chuang BIB008 . A real time gaze tracking method robust to variable illumination levels and driver wearing glasses in an automotive environment is reported in BIB001 . It uses Kalman filtering and mean-shift method to track driver eyes based on their locations in a previous frame. SVM with linear, polynomial and Gaussian kernels are used for eye verification. Park BIB006 proposed a robust SVM based driver gaze zone estimation system that works during day and night conditions and is robust to driver wearing eyeglasses. A multiclass SVM constructed from multiple two class (binary) SVMs was constructed to classify 18 driver gaze zones using a large database of 18000 gaze feature vectors. Another SVM based gaze zone classifier that takes face parts, i.e., mouth, eyes, and nose locations is reported in Chuang BIB008 . A multiclass linear SVM is used which inputs the feature descriptor to output 8 gaze directions. Use of other type of classifiers is reported in Tawari BIB009 and Kwak BIB007 . In BIB009 a distributed camera setup is used for estimating gaze zones combined with head pose dynamics. A random forest classifier is used with static and dynamic features (head pose angles and time series statistics) to classify 8 gaze zones for the driver. In BIB007 Viola Jones method is used to detect the face followed by using linear discriminant analysis (LDA) to extract features from the eye region for classification. k-nearest neighbour method with Euclidean norm is used to classify the obtained features into seven gaze directions. Gaze estimation in automotive using Purkinje images and PCCR methods are reported in Yang  , Salvucci  , Batista  ,Choi BIB010 and Ji BIB002 . Several papers report the application of gaze information along with other facial and visual parameters for derivation of driver psychological/cognitive state. Key works include Bergasa BIB003 which reports a method for driver vigilance estimation using fuzzy classifier taking six parameters: Percent eye closure (PERCLOS), eye closure duration, blink frequency, nodding frequency, face position, and fixed gaze. A dynamic Bayesian Neural network approach is used in Yang BIB004 for driver fatigue detection using gaze location along with face detection, eye positioning and iris tracking. Ji BIB002 combines features such as eyelid and head movement, gaze, and facial expressions with a Bayesian network to determine driver fatigue. Another important work that considers eye movements for deriving driver cognitive state is BIB005 . It takes into consideration eye movement features like fixations, saccades, and smooth pursuits and calculates 16 different eye movement features. These are then fed to an SVM, static and dynamic Bayesian networks to do their performance comparison for prediction of driver distraction state. BIB012 presents a low cost system to detect eyes-off-the road condition of a driver using facial feature tracking, 3D head pose and gaze estimation. A monocular camera is installed close to the steering wheel for tracking a driver's facial landmarks and accurate estimation of driver pose, location and gaze direction. Then, with 3D analysis of car/driver geometry the driver's eyes off the road condition is predicted in real time. In BIB014 , driver gaze behaviour is studied to evaluate driving performance of a user when they had to interact with a portable navigation system while driving. Glance frequency and glance time were estimated to study impact of varying display sizes and positions of the navigation device while in use during driving. A portable driver assistance system involving driver drowsiness detection and eye gaze tracking is implemented using a Raspberry Pi and machine vision algorithms in BIB011 . A new multi-depth calibration approach is presented in [131] for obtaining 3D user PoG with stereo face cameras and monocular scene cameras for performing driver intent and actions prediction. Several works study the dynamics between head pose and gaze behavior of drivers. In BIB015 the relationship between gaze and head pose are studied and regression models are developed to predict gaze location from the position and orientation of a driver's head. In BIB013 head and eye poses of 40 driving participants are estimated from monocular video and relative significance of head vs eye movements in gaze classification is studied. It is observed that driver behavior can be grouped into two cases, i.e., when the head moves a lot and gaze classification is mostly affected by head pose. The other case is when the head stays still and only the eyes move and thus accurate classification requires study of eye pose. 