Effective and real-time eyeblink detection is of wide-range applications, such as deception detection, drive fatigue detection, face anti-spoofing, etc. Although numerous of efforts have already been paid, most of them focus on addressing the eyeblink detection problem under the constrained indoor conditions with the relative consistent subject and environment setup. Nevertheless, towards the practical applications eyeblink detection in the wild is more required, and of greater challenges. However, to our knowledge this has not been well studied before. In this paper, we shed the light to this research topic. A labelled eyeblink in the wild dataset (i.e., HUST-LEBW) of 673 eyeblink video samples (i.e., 381 positives, and 292 negatives) is first established by us. These samples are captured from the unconstrained movies, with the dramatic variation on human attribute, human pose, illumination condition, imaging configuration, etc. Then, we formulate eyeblink detection task as a spatial-temporal pattern recognition problem. After locating and tracking human eye using SeetaFace engine and KCF tracker respectively, a modified LSTM model able to capture the multi-scale temporal information is proposed to execute eyeblink verification. A feature extraction approach that reveals appearance and motion characteristics simultaneously is also proposed. The experiments on HUST-LEBW reveal the superiority and efficiency of our approach. It also verifies that, the existing eyeblink detection methods cannot achieve satisfactory performance in the wild.
Besides the patter recognition model, another essential issue for eyeblink verification is feature extraction. Generally speaking, appearance feature (e.g., EAR @cite_19 , LBP @cite_40 , Haar @cite_34 , or HOG @cite_37 ) or motion feature (e.g., KTL tracker motion @cite_19 or pixel-wise frame difference between the consecutive 2 frames @cite_30 ) are extracted to this end. Nevertheless, few approaches take appearance and motion information into consideration simultaneously. To address this, we propose to use uniform LBP as appearance feature and its difference between the consecutive 2 frames as motion feature to jointly characterize eyeblink.
@cite_30 @cite_37 @cite_19 @cite_40 @cite_34
1979845503 2161969291 2111119856 2134637340
A vision-based human--computer interface is presented in the paper. The interface detects voluntary eye-blinks and interprets them as control commands. The employed image processing methods include Haar-like features for automatic face detection, and template matching based eye tracking and eye-blink detection. Interface performance was tested by 49 users (of which 12 were with physical disabilities). Test results indicate interface usefulness in offering an alternative mean of communication with computers. The users entered English and Polish text (with average time of less than 12s per character) and were able to browse the Internet. The interface is based on a notebook equipped with a typical web camera and requires no extra light sources. The interface application is available on-line as open-source software. We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds. This paper proposes a robust and efficient eye state detection method based on an improved algorithm called LBP+SVM mode. LBP (local binary pattern) methodology is first used to select the two groups of candidates from a whole face image. Then corresponding SVMs (supporting vector machine) are employed to verify the real eye and its state. The LBP methodology makes it robust against rotation, illumination and occlusion to find the candidates, and the SVM helps to make the final verification correct. In this paper, we present an approach for eye state recognition and closed-eye photo correction. For eye state recognition, AdaBoosted cascade open-eye detectors of different scales are trained. For closed-eye photo correction, a PCA generative model of concatenated corresponding closed-eye and open-eye texture patterns is built, and given a closed-eye texture pattern, an algorithm is proposed to recover its corresponding open-eye one for closed-eye replacement. Experiments on popular consumer images show our open-eye detectors achieved 94.71 correct recognition rate, and the closed-eye photo correction looks very natural.