Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Automatic face recognition systems have made great strides. They still, however cannot cope with changes due to lighting and cannot detect disguises, Both of these issues are critical for the employment of face recognition systems in high security applications, such as embassy perimeters, federal plazas, mid the like. We propose novel imaging solutions that address these difficult problems. We demonstrate with theoretical and experimental arguments that a dual-band fusion system in the near infrared can segment human faces much more accurately than traditional visible band face detection systems. Face detection is useful by itself as an early warning method in certain surveillance applications. Accurate face delineation can also improve the performance of face recognition systems in certain difficult scenarios, particularly in outside environments. We also demonstrate with theoretical and experimental arguments that the upper band of the near infrared (1.4-2.4 /spl mu/m) is particularly advantageous for disguise detection purposes. This is attributable to the unique and universal properties of the human skin in this sub-band. We conclude the paper with a description of our ongoing and future efforts. <s> BIB001 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> This survey focuses on recognition performed by matching models of the three-dimensional shape of the face, either alone or in combination with matching corresponding two-dimensional intensity images. Research trends to date are summarized, and challenges confronting the development of more accurate three-dimensional face recognition are identified. These challenges include the need for better sensors, improved recognition algorithms, and more rigorous experimental methodology. <s> BIB002 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> In this paper we show the potential of two new features as powerful anti-spoofing measures for face-voice person authentication systems. The features based on latent semantic analysis (LSA) and canonical correlation analysis (CCA), enhance the performance of the authentication system in terms of better anti-imposture abilities and guard against video replay attacks, which is a challenging type of spoof attack. Experiments conducted on 2 speaking-face databases, VidTIMIT and UCBN, show around 42% improvement in error rate with CCA features and 61% improvement with LSA features over feature-level fusion of face-voice feature vectors. <s> BIB003 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> The current dominant approaches to face recognition rely on facial characteristics that are on or over the skin. Some of these characteristics have low permanency can be altered, and their phenomenology varies significantly with environmental factors (e.g., lighting). Many methodologies have been developed to address these problems to various degrees. However, the current framework of face recognition research has a potential weakness due to its very nature. We present a novel framework for face recognition based on physiological information. The motivation behind this effort is to capitalize on the permanency of innate characteristics that are under the skin. To establish feasibility, we propose a specific methodology to capture facial physiological patterns using the bioheat information contained in thermal imagery. First, the algorithm delineates the human face from the background using the Bayesian framework. Then, it localizes the superficial blood vessel network using image morphology. The extracted vascular network produces contour shapes that are characteristic to each individual. The branching points of the skeletonized vascular network are referred to as thermal minutia points (TMPs) and constitute the feature database. To render the method robust to facial pose variations, we collect for each subject to be stored in the database five different pose images (center, midleft profile, left profile, midright profile, and right profile). During the classification stage, the algorithm first estimates the pose of the test image. Then, it matches the local and global TMP structures extracted from the test image with those of the corresponding pose images in the database. We have conducted experiments on a multipose database of thermal facial images collected in our laboratory, as well as on the time-gap database of the University of Notre Dame. The good experimental results show that the proposed methodology has merit, especially with respect to the problem of low permanence over time. More importantly, the results demonstrate the feasibility of the physiological framework in face recognition and open the way for further methodological and experimental research in the area <s> BIB004 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> A robust face detection technique along with mouth localization, processing every frame in real time (video rate), is presented. Moreover, it is exploited for motion analysis onsite to verify "liveness" as well as to achieve lip reading of digits. A methodological novelty is the suggested quantized angle features ("quangles") being designed for illumination invariance without the need for preprocessing (e.g., histogram equalization). This is achieved by using both the gradient direction and the double angle direction (the structure tensor angle), and by ignoring the magnitude of the gradient. Boosting techniques are applied in a quantized feature space. A major benefit is reduced processing time (i.e., that the training of effective cascaded classifiers is feasible in very short time, less than 1 h for data sets of order 104). Scale invariance is implemented through the use of an image scale pyramid. We propose "liveness" verification barriers as applications for which a significant amount of computation is avoided when estimating motion. Novel strategies to avert advanced spoofing attempts (e.g., replayed videos which include person utterances) are demonstrated. We present favorable results on face detection for the YALE face test set and competitive results for the CMU-MIT frontal face test set as well as on "liveness" verification barriers. <s> BIB005 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> This paper addresses the subject of liveness detection, which is a test that ensures that biometric cues are acquired from a live person who is actually present at the time of capture. The liveness check is performed by measuring the degree of synchrony between the lips and the voice extracted from a video sequence. Three new methods for asynchrony detection based on co-inertia analysis (CoIA) and a fourth based on coupled hidden Markov models (CHMMs) are derived. Experimental comparisons are made with several methods previously used in the literature for asynchrony detection and speaker location. The reported results demonstrate the effectiveness and superiority of the proposed new methods based on both CoIA and CHMMs as asynchrony detection methods. <s> BIB006 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Resisting spoofing attempts via photographs and video playbacks is a vital issue for the success of face biometrics. Yet, the ldquolivenessrdquo topic has only been partially studied in the past. In this paper we are suggesting a holistic liveness detection paradigm that collaborates with standard techniques in 2D face biometrics. The experiments show that many attacks are avertible via a combination of anti-spoofing measures. We have investigated the topic using real-time techniques and applied them to real-life spoofing scenarios in an indoor, yet uncontrolled environment. <s> BIB007 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> This research presents a novel 2D feature space where real faces and masked fake faces can be effectively discriminated. We exploit the reflectance disparity based on albedo between real faces and fake materials. The feature vector used consists of radiance measurements of the forehead region under 850 and 685 nm illuminations. Facial skin and mask material show linearly separable distributions in the feature space proposed. By simply applying Fisher's linear discriminant, we have achieved 97.78% accuracy in fake face detection. Our method can be easily implemented in commercial face verification systems. <s> BIB008 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Existing face liveness detection algorithms adopt behavioural challenge-response methods that require user cooperation. To be verified live, users are expected to obey some user unfriendly requirement. In this paper, we present a multispectral face liveness detection method, which is user cooperation free. Moreover, the system is adaptive to various user-system distances. Using the Lambertian model, we analyze multispectral properties of human skin versus non-skin, and the discriminative wavelengths are then chosen. Reflectance data of genuine and fake faces at multi-distances are selected to form a training set. An SVM classifier is trained to learn the multispectral distribution for a final Genuine-or-Fake classification. Compared with previous works, the proposed method has the following advantages: (a) The requirement on the users' cooperation is no longer needed, making the liveness detection user friendly and fast. (b) The system can work without restricted distance requirement from the target being analyzed. Experiments are conducted on genuine versus planar face data, and genuine versus mask face data. Furthermore a comparison with the visible challenge-response liveness detection method is also given. The experimental results clearly demonstrate the superiority of our method over previous systems. <s> BIB009 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Face liveness detection in visible light (VIS) spectrum is facing great challenges. Beyond visible light spectrum, thermal IR (TIR) has intrinsic live signal itself. In this paper, we present a novel liveness detection approach based on thermal IR spectrum. Live face is modeled in the cross-modality of thermal IR and visible light spectrum. In our model, canonical correlation analysis between visible and thermal IR face is exploited. The correlation of different face parts is also investigated to illustrate more correlative features and be helpful to improve live face detection ability. An extensive set of liveness detection experiments are presented to show effectiveness of our approach and other correlation methods are also tested for comparison. <s> BIB010 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> The recognition of faces in unconstrained environments is a challenging problem. The aim of this work is to carry out a comparative study of face recognition methods working in the thermal spectrum (8-12@mm) that are suitable for working properly in these environments. The analyzed methods were selected by considering their performance in former comparative studies, in addition to being real-time, to requiring just one image per person, and to being fully online (no requirements of offline enrollment). Thus, in this study three local-matching methods based on histograms of Local Binary Pattern (LBP) features, on histograms of Weber Linear Descriptors (WLD), and on Gabor Jet Descriptors (GJD), as well as two global image-matching method based on Scale-Invariant Feature Transform (SIFT) Descriptors, and Speeded Up Robust Features (SURF) Descriptors, are analyzed. The methods are compared using the Equinox and UCHThermalFace databases. The use of these databases allows evaluating the methods in real-world conditions that include natural variations in illumination, indoor/outdoor setup, facial expression, pose, accessories, occlusions, and background. The UCHThermalFace database is described for the first time in this article and WLD is used for the first time in face recognition. The results of this comparative study are intended to be a guide for developers of face recognition systems. The main conclusions of this study are: (i) all analyzed methods perform very well under the conditions in which they were evaluated, except for the case of GJD that has low performance in outdoor setups; (ii) the best tradeoff between high recognition rate and fast processing speed is obtained by WLD-based methods, although the highest recognition rate in all cases is obtained by SIFT-based methods; and (iii) in experiments where the test images are acquired in an outdoor setup and the gallery images are acquired in an indoor setup, or vice versa, the performance of all evaluated methods is very low. As part of the future work, the use of normalization algorithms and calibration procedures in order to tackle this last issue will be analyzed. <s> BIB011 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> In this paper, an efficient approach for human face recognition based on the use of minutiae points in thermal face image is proposed. The thermogram of human face is captured by thermal infra-red camera. Image processing methods are used to pre-process the captured thermogram, from which different physiological features based on blood perfusion data are extracted. Blood perfusion data are related to distribution of blood vessels under the face skin. In the present work, three different methods have been used to get the blood perfusion image, namely bit-plane slicing and medial axis transform, morphological erosion and medial axis transform, sobel edge operators. A five layer feed-forward back propagation neural network is used as the classification tool. It has been found that the first method supercedes the other two producing an accuracy of 97.62% with block size 16×16 for bit-plane 4. <s> BIB012 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Photographs, videos or masks can be used to spoof face recognition systems. In this paper, a countermeasure is proposed to protect face recognition systems against 3D mask attacks. The reason for the lack of studies on countermeasures against mask attacks is mainly due to the unavailability of public databases dedicated to mask attack. In this study, a 2D+3D mask attacks database is used that is prepared for a research project in which the authors are all involved. The proposed countermeasure is based on the fusion of the information extracted from both the texture and the depth images in the mask database, and provides satisfactory results to protect recognition systems against mask attacks. Another contribution of this study is that the countermeasure is integrated to the selected baseline systems for 2D and 3D face recognition, which provides to analyze the performances of the systems with/without attacks and with/without the countermeasure. <s> BIB013 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique is proposed to protect face recognition systems against mask attacks. To the best of our knowledge, this is the first time a countermeasure is proposed to detect mask attacks. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, a 2D+3D face mask attacks database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated on both the texture images and the depth maps, separately. The results show that the proposed countermeasure gives satisfactory results using both the texture images and the depth maps. The performance of the countermeasure is observed to be slight better when the technique is applied on texture images instead of depth maps, which proves that face texture provides more information than 3D face shape characteristics using the proposed approach. <s> BIB014 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Face photographs, videos or masks can be used to spoof face recognition systems. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique, which analyzes the reflectance characteristics of masks and real faces, is proposed to detect mask attacks. There are limited studies on countermeasures against mask attacks. The reason for this delay is mainly due to the unavailability of public mask attack databases. In this study, a 2D+3D face mask attack database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated using the texture images which were captured during the acquisition of 3D scans. The results of the proposed countermeasure outperform the results of existing techniques, achieving a classification accuracy of 94.47%. In this paper, it is also proved that reflectance analysis may provide more information for the purpose of mask spoofing detection compared to texture analysis. <s> BIB015 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> 3) FACE ANTI-SPOOFING: SENSOR LEVEL APPROACHES <s> Face verification, though for humans seems to be an easy task, is a long-standing research area. With challenging covariates such as disguise or face obfuscation, automatically verifying the identity of a person is assumed to be very hard. This paper explores the feasibility of face verification under disguise variations using multi-spectrum (visible and thermal) face images. We propose a framework, termed as Aravrta1, which classifies the local facial regions of both visible and thermal face images into biometric (regions without disguise) and non-biometric (regions with disguise) classes. The biometric patches are then used for facial feature extraction and matching. The performance of the algorithm is evaluated on the IHTD In and Beyond Visible Spectrum Disguise database that is prepared by the authors and contains images pertaining to 75 subjects with different kinds of disguise variations. The experimental results suggest that the proposed framework improves the performance compared to existing algorithms, however there is a need for more research to address this important covariate. <s> BIB016
Regarding sensor-level anti-spoofing techniques, the number of contributions is still not comparable to that of softwarebased approaches. However, some interesting methods have been proposed base on imaging technology outside the visual spectrum, such as: complementary infrared (IR) or near infrared (NIR) images, which are even claimed to provide sufficient information to distinguish between identical twins BIB001 ,  ; comparing the reflectance information of real faces and fake materials using a specific set-up of LEDs and photodiodes at two different wavelengths BIB008 , BIB009 . In addition to the previous works, there are other technologies, initially proposed for personal authentication purposes, that could also be used as sensor-level anti-spoofing techniques. Although in most cases no rigorous study has yet been carried out regarding their performance under liveness detection scenarios, such potentially useful mechanisms for face anti-spoofing would include: thermal imaging BIB004 , BIB011 , detection of the facial vein pattern BIB012 , or 3D face acquisition BIB002 . For instance, 3D sensors can be very robust against attacks carried out with flat surfaces (e.g., photo or video attacks) as almost no depth difference is detected compared to real faces. On the other hand, their performance under mask-attacks has just started to be investigated using texture analysis inspired on 2D protection methods based on LBPs BIB013 , BIB014 and also on the analysis of the reflectance components that can be computed from 3D scans BIB015 . Similarly, systems based on the detection of the face thermogram would be, in principle, very accurate detecting all three types of major face spoofing attacks (i.e., photo, video and mask attacks), as no thermal difference is expected in fake faces. Some initial efforts to study thermal imaging for liveness detection have already been carried out BIB010 , including the acquisition of a significantly large database of thermal images for standard and disguised access attempts where very promising results have been obtained BIB016 . Still, it would be necessary to capture further face thermogram data under normal operational conditions and under spoofing attacks in order to validate these initial findings. The fact that these techniques (i.e., 3D and thermal face recognition) already present solid backgrounds for personal authentication, can become an added advantage for their development as security enhancing alternatives, since both tasks (i.e., recognition and anti-spoofing) could be performed from the same sample. Multimodality has also been explored as a sensorlevel liveness detection approach. Many multibiometric anti-spoofing techniques consider the combination of face and voice, as they are two easily measurable traits. Such methods exploit the correlation between the lips movement and the speech being produced BIB003 ,  - BIB006 or use specific information obtained from the lip movement in the utterance of a preassigned pin code BIB005 . The latter method could also be classified in the challengeresponse category, as the user is asked to respond to a system command. Other studied challenge-response strategies consider voluntary eye blinking and mouth movement following a request from the system BIB007 (included in the present survey among the feature-level dynamic-based approaches, see Sect. III-B.1).