Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> This paper presents a system for analyzing human driver visual attention. The system relies on estimation of global motion and color statistics to robustly track a person's head and facial features. The system is fully automatic, it can initialize automatically, and reinitialize when necessary. The system classifies rotation in all viewing directions, detects eye/mouth occlusion, detects eye blinking and eye closure, and recovers the three dimensional gaze of the eyes. In addition, the system is able to track both through occlusion due to eye blinking, and eye closure, large mouth movement, and also through occlusion due to rotation. Even when the face is fully occluded due to rotation, the system does not break down. Further the system is able to track through yawning, which is a large local mouth motion. Finally, results are presented, and future work on how this system can be used for more advanced driver visual attention monitoring is discussed. <s> BIB001 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> We present a non-intrusive prototype computer vision system for real-time monitoring driver's vigilance. It is based on a hardware system, for real time acquisition of driver's images using an active IR illuminator, and their software implementation for monitoring some visual behaviours that characterize a driver's level of vigilance. These are the eyelid movements and the pose face. The system has been tested with different sequences recorded on night and day driving conditions in a motorway and with different users. We show some experimental results and some conclusions about the performance of the system. <s> BIB002 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> We ran a randomized cross-over design study under sleep-deprived and non-sleep-deprived driving conditions to test the effects of sleep restriction on real driving performance. The study was performed in a sleep laboratory and on an open French highway. Twenty-two healthy male subjects (age = 21.5 +/- 2 years; distance driven per year = 12,225 +/- 4739 km (7641 +/- 2962 miles) [mean +/- S.D.]) drove 1000 km (625 miles) over 10 h during five 105 min sessions on an open highway. Self-rated fatigue and sleepiness before each session, number of inappropriate line crossings from video recordings and simple reaction time (RT) were measured. Total crossings increased after sleep restriction (535 crossings in the sleep-restricted condition versus 66 after non-restricted sleep (incidence rate ratio (IRR): 8.1; 95% confidence interval (95% CI): 3.2-20.5; p < 0.001)), from the first driving session. The interaction between the two factors (conditionxtime of day) was also significant (F(5, 105) = 3.229; p < 0.05). Increasing sleepiness score was associated with increasing crossings during the next driving session in the sleep-restricted (IRR: 1.9; 95% CI: 1.4-2.4) but not in the non-restricted condition (IRR: 1.0; 95% CI: 0.8-1.3). Increasing self-perceived fatigue was not associated with increasing crossings in either condition (IRR: 0.95; 95% CI: 0.93-0.98 and IRR: 1.0; 95% CI: 0.98-1.02). Rested subjects drove 1000 km with four shorts breaks with only a minor performance decrease. Sleep restriction induced important performance degradation even though time awake (8h) and session driving times (105 min) were relatively short. Major inter-individual differences were observed under sleep restriction. Performance degradation was associated with sleepiness and not fatigue. Sleepiness combined with fatigue significantly affected RT. Road safety campaigns should encourage drivers to avoid driving after sleep restriction, even on relatively short trips especially if they feel sleepy. <s> BIB003 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> The advance of computing technology has provided the means for building intelligent vehicle systems. Drowsy driver detection system is one of the potential applications of intelligent vehicle systems. Previous approaches to drowsiness detection primarily make preassumptions about the relevant behavior, focusing on blink rate, eye closure, and yawning. Here we employ machine learning to datamine actual human behavior during drowsiness episodes. Automatic classifiers for 30 facial actions from the Facial Action Coding system were developed using machine learning on a separate database of spontaneous expressions. These facial actions include blinking and yawn motions, as well as a number of other facial movements. In addition, head motion was collected through automatic eye tracking and an accelerometer. These measures were passed to learning-based classifiers such as Adaboost and multinomial ridge regression. The system was able to predict sleep and crash episodes during a driving computer game with 96% accuracy within subjects and above 90% accuracy across subjects. This is the highest prediction rate reported to date for detecting real drowsiness. Moreover, the analysis revealed new information about human behavior during drowsy driving. <s> BIB004 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> To improve driving safety,the authors propose an approach to locate a driver's mouth by a web camera and extract texture features from mouth comers for monitoring drivers' yawning.Firstly,it detects drivers' left and right mouth corners by gray projection based on the result of driver face detection,and then it extracts texture features of drivers' mouth corners by Gabor wavelets.Finally,LDA is used to classify Gabor features for yawning detection.The proposed approach is tested on 3 000 images from thirty subjects with variations in illuminations,poses,and facial accessories(glasses).Yawning is also detected by the ratio of mouth height to width as a baseline.Experiment results show that the proposed approach is suitable for real time yawning detection,Gabor features are more powerful than geometric features for yawning representation,and an average recognition rate of 91.97%is achieved which is much better than the baseline. <s> BIB005 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> Driver fatigue is a significant factor in many traffic accidents. We propose a novel approach for driver fatigue detection from facial image sequences, which is based on multiscale dynamic features. First, Gabor filters are used to get a multiscale representation for image sequences. Then Local Binary Patterns are extracted from each multiscale image. To account for the temporal aspect of human fatigue, the LBP image sequence is divided into dynamic units, and a histogram of each dynamic unit is computed and concatenated as dynamic features. Finally a statistical learning algorithm is applied to extract the most discriminative features from the multiscale dynamic features and construct a strong classifier for fatigue detection. The proposed approach is validated under real-life fatigue conditions. The test data includes 600 image sequences with illumination and pose variations from 30 people's videos. Experimental results show the validity of the proposed approach, and a correct rate of 98.33% is achieved which is much better than the baselines. <s> BIB006 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> In this paper, we consider the problem of estimating the pose of a driver from video data. We propose to use isophote features to improve classification performance when illumination varies. In particular, we both use the direction and curvature features. The features are encoded into direction histogram and curvature histogram in the blocks of the image. Experimental results show that the proposed features could well describe the structure of driver face image, and have a good performance in real scene. <s> BIB007 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> Fatigue driving easily causes traffic accidents. drowsiness is an important hint of Fatigue. A novel drowsiness detection algorithms based on eyelid movement is proposed in the paper. The cascaded classifiers algorithm is used to detect driver's face and the diamond searching algorithm used to trace the face. A simple feature is then extracted from temporal difference image and used to analyze rules of eyelid movement in drowsiness. Furthermore, three criterions are also presented and used to judge whether a driver is drowsy or not. Experimental results show that this new algorithm achieves a satisfied performance for drowsiness detection. <s> BIB008 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> A new scheme for driver fatigue detection is presented, which is based on the nonlinear unscented Kalman filter and eye tracking. Assuming a probability distribution than to approximate an arbitrary nonlinear function or transformation, eye nonlinear tracking can be achieved using an unscented transformation (UT), which adopts a set of deterministic sigma points to match the posterior probability density function of the eye movement. Driver fatigue can be detected using the percentage of eye closure (PERCLOS) framework in a realistic driving condition after the eye nonlinear tracking. This system was tested adequately in realistic driving environments with subjects of different genders, with/without glasses, in day/night driving, being commercial/noncommercial drivers, in continuous driving time, and under different road conditions. The last experimental results show that the proposed method not only improves the robustness for nonlinear eye tracking, but also can provide more accurate estimation than the traditional Kalman filter. <s> BIB009 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> Driver distraction and inattention are prominent causes of automotive collisions. To enable driver-assistance systems to address these problems, we require new sensing approaches to infer a driver's focus of attention. In this paper, we present a new procedure for static head-pose estimation and a new algorithm for visual 3-D tracking. They are integrated into the novel real-time (30 fps) system for measuring the position and orientation of a driver's head. This system consists of three interconnected modules that detect the driver's head, provide initial estimates of the head's pose, and continuously track its position and orientation in six degrees of freedom. The head-detection module consists of an array of Haar-wavelet Adaboost cascades. The initial pose estimation module employs localized gradient orientation (LGO) histograms as input to support vector regressors (SVRs). The tracking module provides a fine estimate of the 3-D motion of the head using a new appearance-based particle filter for 3-D model tracking in an augmented reality environment. We describe our implementation that utilizes OpenGL-optimized graphics hardware to efficiently compute particle samples in real time. To demonstrate the suitability of this system for real driving situations, we provide a comprehensive evaluation with drivers of varying ages, race, and sex spanning daytime and nighttime conditions. To quantitatively measure the accuracy of system, we compare its estimation results to a marker-based cinematic motion-capture system installed in the automotive testbed. <s> BIB010 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> Every year, traffic accidents due to human errors cause increasing amounts of deaths and injuries globally. To help reduce the amount of fatalities, in the paper presented here, a new module for Advanced Driver Assistance System (ADAS) which deals with automatic driver drowsiness detection based on visual information and Artificial Intelligence is presented. The aim of this system is to locate, track, and analyze both the drivers face and eyes to compute a drowsiness index, where this real-time system works under varying light conditions (diurnal and nocturnal driving). Examples of different images of drivers taken in a real vehicle are shown to validate the algorithms used. <s> BIB011 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> The purpose of this study was to evaluate commercially available devices for driver fatigue monitoring with particular focus on the needs of the mining industry. We present an overview of fatigue monitoring technologies (FMTs) and propose means to evaluate the devices. Three video-based devices were selected and used in an overnight driving simulation study to test their accuracy. In total 14 healthy volunteers performed the driving simulation tasks in eight test runs separated by breaks of approximately 10 min. EEG and EOG were recorded during the driving periods. The output variable of the FMT devices (percentage of eye closures, PERCLOS), subjectively rated fatigue on the Karolinska sleepiness scale (KSS), and driving performance in terms of standard deviation of lateral position in lane (SDL) were also recorded throughout testing sessions. Regression analysis revealed that PERCLOS is significantly related to higher KSS scores and to SDL. Calculations at a finer temporal resolution as well as on an intra-subject level showed decreased correlation coefficients. Discriminant analysis of PERCLOS and EEG/EOG suggested that PERCLOS does not differentiate well between mild and strong fatigue. The results suggest that under laboratory conditions current FMT devices are reliable when temporal resolution is not too fine (>30 min) and data averaged across several subjects is utilized, but fail to give a valid prediction of subjective fatigue as well as of driving performance on an individual level. <s> BIB012 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> Abstract Several researchers have investigated the relation between vigilance and ocular variables such as saccade, slow eye movement, pupil, blink, or eyelid closure. This study was undertaken to find the most effective indicator among these ocular variables for evaluating short-term (1 min) fluctuation of vigilance, and to investigate the ability of the most effective ocular variable for predicting deteriorated vigilance during behavioral maintenance of the wakefulness test (Oxford sleep resistance test: OSLER test). Nine healthy volunteers (two women, 19–30 years old, 23.4 ± 3.9 years old) participated in this study. Ocular variables were recorded during the OSLER test at 10 A.M. and 2 P.M. before and after partial sleep deprivation (4 h sleep). The periods during the OSLER test were divided into 1 min epochs. Each epoch was classified according to the number of consecutive missed responses. Decreased blink frequency and pupil diameter as well as increased percentage of eyelid closure time (PERCLOS) and slow eye movement were observed as the consecutive missed responses increased. Among these variables, PERCLOS showed the highest ability to detect occurrence of any missed response and three or more consecutive missed responses. Moreover, a missed response seldom occurred (0.2 ± 0.2/20 trial/min) when PERCLOS was less than 11.5% per minute. Results suggest that, among the ocular variables, PERCLOS can prevent error or accident caused by low vigilance most effectively. <s> BIB013 </s> Detecting Driver Drowsiness Based on Sensors: A Review <s> Behavioral Measures <s> OBJECTIVES ::: This study evaluated oculometrics as a detector of fatigue in Air Force-relevant tasks after sleep deprivation. Using the metrics of total eye closure duration (PERCLOS) and approximate entropy (ApEn), the relation between these eye metrics and fatigue-induced performance decrements was investigated. ::: ::: ::: BACKGROUND ::: One damaging effect to the successful outcome of operational military missions is that attributed to sleep deprivation-induced fatigue. Consequently, there is interest in the development of reliable monitoring devices that can assess when an operator is overly fatigued. ::: ::: ::: METHOD ::: Ten civilian participants volunteered to serve in this study. Each was trained on three performance tasks: target identification, unmanned aerial vehicle landing, and the psychomotor vigilance task (PVT). Experimental testing began after 14 hr awake and continued every 2 hr until 28 hr of sleep deprivation was reached. ::: ::: ::: RESULTS ::: Performance on the PVT and target identification tasks declined significantly as the level of sleep deprivation increased.These performance declines were paralleled more closely by changes in the ApEn compared to the PERCLOS measure. ::: ::: ::: CONCLUSION ::: The results provide evidence that the ApEn eye metric can be used to detect fatigue in relevant military aviation tasks. ::: ::: ::: APPLICATION ::: Military and commercial operators could benefit from an alertness monitoring device. <s> BIB014
A drowsy person displays a number of characteristic facial movements, including rapid and constant blinking, nodding or swinging their head, and frequent yawning BIB005 . Computerized, non-intrusive, behavioral approaches are widely used for determining the drowsiness level of drivers by measuring their abnormal behaviors BIB004 . Most of the published studies on using behavioral approaches to determine drowsiness, focus on blinking BIB002  BIB008 . PERCLOS (which is the percentage of eyelid closure over the pupil over time, reflecting slow eyelid closures, or "droops", rather than blinks) has been analyzed in many studies BIB009  BIB013 BIB014 . This measurement has been found to be a reliable measure to predict drowsiness  and has been used in commercial products such as Seeing Machines  and Lexus  . Some researchers used multiple facial actions, including inner brow rise, outer brow rise, lip stretch, jaw drop and eye blink, to detect drowsiness BIB006 BIB004 . However, research on using other behavioral measures, such as yawning BIB001 and head or eye position orientation BIB010 BIB007 , to determine the level of drowsiness is ongoing ( Table 2) . The main limitation of using a vision-based approach is lighting. Normal cameras do not perform well at night BIB002 . In order to overcome this limitation, some researchers have used active illumination utilizing an infrared Light Emitting Diode (LED) BIB002 . However, although these work fairly well at night, LEDs are considered less robust during the day  . In addition, most of the methods have been tested on data obtained from drivers mimicking drowsy behavior rather than on real video data in which the driver gets naturally drowsy. Mostly, image is acquired using simple CCD or web camera during day  and IR camera during night BIB011 at around 30 fps. After capturing the video, some techniques, including Connected Component Analysis, Cascade of Classifiers or Hough Transform, Gabor Filter, Haar Algorithm are applied to detect the face, eye or mouth BIB009 BIB004  BIB011 . After localizing the specific region of interest within the image, features such as PERCLOS, yawning frequency and head angle, are extracted using an efficient feature extraction technique, such as Wavelet Decomposition, Gabor Wavelets, Discrete Wavelet Transform or Condensation Algorithm BIB005 BIB004  BIB011 . The behavior is then analyzed and classified as either normal, slightly drowsy, highly drowsy through the use of classification methods such as support vector machine, fuzzy classifier, neural classifier and linear discriminant analysis BIB005 BIB004 BIB002  . However, it has been found that the rate of detecting the correct feature, or the percentage of success among a number of detection attempts, varies depending on the application and number of classes. The determination of drowsiness using PERCLOS and Eye Blink has a success rate of close to 100% BIB002 and 98% BIB008 , respectively. However it has to be noted that, the high positive detection rate achieved by BIB002 was when the subjects didn't wear glasses. Likewise, as most researchers conducted their experiments in simulated environment they achieved a higher success rate. The positive detection rate decreased significantly when the experiment was carried out in a real environment BIB003 . Another limitation of behavioral measure was brought out in an experiment conducted by Golz et al. They evaluated various drowsiness monitoring commercial products, and observed that driver state cannot be correlated to driving performance and vehicle status based on behavioral measures alone BIB012 .