Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> The recognizer discussed will automatically recognize telephone‐quality digits spoken at normal speech rates by a single individual, with an accuracy varying between 97 and 99 percent. After some preliminary analysis of the speech of any individual, the circuit can be adjusted to deliver a similar accuracy on the speech of that individual. The circuit is not, however, in its present configuration, capable of performing equally well on the speech of a series of talkers without recourse to such adjustment.Circuitry involves division of the speech spectrum into two frequency bands, one below and the other above 900 cps. Axis‐crossing counts are then individually made of both band energies to determine the frequency of the maximum syllabic rate energy with each band. Simultaneous two‐dimensional frequency portrayal is found to possess recognition significance. Standards are then determined, one for each digit of the ten‐digit series, and are built into the recognizer as a form of elemental memory. By means of... <s> BIB001 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> This research presents a novel 2D feature space where real faces and masked fake faces can be effectively discriminated. We exploit the reflectance disparity based on albedo between real faces and fake materials. The feature vector used consists of radiance measurements of the forehead region under 850 and 685 nm illuminations. Facial skin and mask material show linearly separable distributions in the feature space proposed. By simply applying Fisher's linear discriminant, we have achieved 97.78% accuracy in fake face detection. Our method can be easily implemented in commercial face verification systems. <s> BIB002 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Advancement and affordability is leading to the popularity of plastic surgery procedures. Facial plastic surgery can be reconstructive to correct facial feature anomalies or cosmetic to improve the appearance. Both corrective as well as cosmetic surgeries alter the original facial information to a large extent thereby posing a great challenge for face recognition algorithms. The contribution of this research is 1) preparing a face database of 900 individuals for plastic surgery, and 2) providing an analytical and experimental underpinning of the effect of plastic surgery on face recognition algorithms. The results on the plastic surgery database suggest that it is an arduous research challenge and the current state-of-art face recognition algorithms are unable to provide acceptable levels of identification performance. Therefore, it is imperative to initiate a research effort so that future face recognition systems will be able to address this important problem. <s> BIB003 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Spoofing with photograph or video is one of the most common manner to circumvent a face recognition system. In this paper, we present a real-time and non-intrusive method to address this based on individual images from a generic webcamera. The task is formulated as a binary classification problem, in which, however, the distribution of positive and negative are largely overlapping in the input space, and a suitable representation space is hence of importance. Using the Lambertian model, we propose two strategies to extract the essential information about different surface properties of a live human face or a photograph, in terms of latent samples. Based on these, we develop two new extensions to the sparse logistic regression model which allow quick and accurate spoof detection. Primary experiments on a large photo imposter database show that the proposed method gives preferable detection performance compared to others. <s> BIB004 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> The increase in twin births has created a requirement for biometric systems to accurately determine the identity ::: of a person who has an identical twin. The discriminability of some of the identical twin biometric traits, ::: such as fingerprints, iris, and palmprints, is supported by anatomy and the formation process of the biometric ::: characteristic, which state they are different even in identical twins due to a number of random factors during ::: the gestation period. For the first time, we collected multiple biometric traits (fingerprint, face, and iris) of ::: 66 families of twins, and we performed unimodal and multimodal matching experiments to assess the ability ::: of biometric systems in distinguishing identical twins. Our experiments show that unimodal finger biometric ::: systems can distinguish two different persons who are not identical twins better than they can distinguish identical ::: twins; this difference is much larger in the face biometric system and it is not significant in the iris biometric ::: system. Multimodal biometric systems that combine different units of the same biometric modality (e.g. multiple ::: fingerprints or left and right irises.) show the best performance among all the unimodal and multimodal biometric ::: systems, achieving an almost perfect separation between genuine and impostor distributions. <s> BIB005 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> A common technique to by-pass 2-D face recognition systems is to use photographs of spoofed identities. Unfortunately, research in counter-measures to this type of attack have not kept-up - even if such threats have been known for nearly a decade, there seems to exist no consensus on best practices, techniques or protocols for developing and testing spoofing-detectors for face recognition. We attribute the reason for this delay, partly, to the unavailability of public databases and protocols to study solutions and compare results. To this purpose we introduce the publicly available PRINT-ATTACK database and exemplify how to use its companion protocol with a motion-based algorithm that detects correlations between the person's head movements and the scene context. The results are to be used as basis for comparison to other counter-measure techniques. The PRINT-ATTACK database contains 200 videos of real-accesses and 200 videos of spoof attempts using printed photographs of 50 different identities. <s> BIB006 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Spoofing face recognition systems with photos or videos of someone else is not difficult. Sometimes, all one needs is to display a picture on a laptop monitor or a printed photograph to the biometric system. In order to detect this kind of spoofs, in this paper we present a solution that works either with printed or LCD displayed photographs, even under bad illumination conditions without extra-devices or user involvement. Tests conducted on large databases show good improvements of classification accuracy as well as true positive and false positive rates compared to the state-of-the-art. <s> BIB007 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Existing 3D face recognition algorithms have achieved high enough performances against public datasets like FRGC v2, that it is difficult to achieve further significant increases in recognition performance. However, the 3D TEC dataset is a more challenging dataset which consists of 3D scans of 107 pairs of twins that were acquired in a single session, with each subject having a scan of a neutral expression and a smiling expression. The combination of factors related to the facial similarity of identical twins and the variation in facial expression makes this a challenging dataset. We conduct experiments using state of the art face recognition algorithms and present the results. Our results indicate that 3D face recognition of identical twins in the presence of varying facial expressions is far from a solved problem, but that good performance is possible. <s> BIB008 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Plastic surgery procedures can significantly alter facial appearance, thereby posing a serious challenge even to the state-of-the-art face matching algorithms. In this paper, we propose a novel approach to address the challenges involved in automatic matching of faces across plastic surgery variations. In the proposed formulation, part-wise facial characterization is combined with the recently popular sparse representation approach to address these challenges. The sparse representation approach requires several images per subject in the gallery to function effectively which is often not available in several use-cases, as in the problem we address in this work. The proposed formulation utilizes images from sequestered non-gallery subjects with similar local facial characteristics to fulfill this requirement. Extensive experiments conducted on a recently introduced plastic surgery database [17] consisting of 900 subjects highlight the effectiveness of the proposed approach. <s> BIB009 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> The matching performance of automated face recognition has significantly improved over the past decade. At the same time several challenges remain that significantly affect the deployment of such systems in security applications. In this work, we study the impact of a commonly used face altering technique that has received limited attention in the biometric literature, viz., non-permanent facial makeup. Towards understanding its impact, we first assemble two databases containing face images of subjects, before and after applying makeup. We present experimental results on both databases that reveal the effect of makeup on automated face recognition and suggest that this simple alteration can indeed compromise the accuracy of a bio-metric system. While these are early results, our findings clearly indicate the need for a better understanding of this face altering scheme and the importance of designing algorithms that can successfully overcome the obstacle imposed by the application of facial makeup. <s> BIB010 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Spoofing attacks are one of the security traits that biometric recognition systems are proven to be vulnerable to. When spoofed, a biometric recognition system is bypassed by presenting a copy of the biometric evidence of a valid user. Among all biometric modalities, spoofing a face recognition system is particularly easy to perform: all that is needed is a simple photograph of the user. In this paper, we address the problem of detecting face spoofing attacks. In particular, we inspect the potential of texture features based on Local Binary Patterns (LBP) and their variations on three types of attacks: printed photographs, and photos and videos displayed on electronic screens of different sizes. For this purpose, we introduce REPLAY-ATTACK, a novel publicly available face spoofing database which contains all the mentioned types of attacks. We conclude that LBP, with ∼15% Half Total Error Rate, show moderate discriminability when confronted with a wide set of attack types. <s> BIB011 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Face antispoofing has now attracted intensive attention, aiming to assure the reliability of face biometrics. We notice that currently most of face antispoofing databases focus on data with little variations, which may limit the generalization performance of trained models since potential attacks in real world are probably more complex. In this paper we release a face antispoofing database which covers a diverse range of potential attack variations. Specifically, the database contains 50 genuine subjects, and fake faces are made from the high quality records of the genuine faces. Three imaging qualities are considered, namely the low quality, normal quality and high quality. Three fake face attacks are implemented, which include warped photo attack, cut photo attack and video attack. Therefore each subject contains 12 videos (3 genuine and 9 fake), and the final database contains 600 video clips. Test protocol is provided, which consists of 7 scenarios for a thorough evaluation from all possible aspects. A baseline algorithm is also given for comparison, which explores the high frequency information in the facial region to determine the liveness. We hope such a database can serve as an evaluation platform for future researches in the literature. <s> BIB012 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Variations in the face appearance caused by plastic surgery on skin texture and geometric structure, can impair the performance of most current face recognition systems. In this work, we proposed to use the Structural Similarity (SSIM) quality map to detect and model variations due to plastic surgeries. In the proposed framework, a S-SIM index weighted multi-patch fusion scheme is developed, where different weights are provided to different patches in accordance with the degree to which each patch may be altered by surgeries. An important feature of the proposed approach, also achieving performance comparable with the current state-of-the-art, is that neither training process is needed nor any background information from other datasets is required. Extensive experiments conducted on a plastic surgery face database demonstrate the potential of SSIM map for matching face images after surgeries. <s> BIB013 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> The problem of detecting face spoofing attacks (presentation attacks) has recently gained a well-deserved popularity. Mainly focusing on 2D attacks forged by displaying printed photos or replaying recorded videos on mobile devices, a significant portion of these studies ground their arguments on the flatness of the spoofing material in front of the sensor. In this paper, we inspect the spoofing potential of subject-specific 3D facial masks for 2D face recognition. Additionally, we analyze Local Binary Patterns based coun-termeasures using both color and depth data, obtained by Kinect. For this purpose, we introduce the 3D Mask Attack Database (3DMAD), the first publicly available 3D spoofing database, recorded with a low-cost depth camera. Extensive experiments on 3DMAD show that easily attainable facial masks can pose a serious threat to 2D face recognition systems and LBP is a powerful weapon to eliminate it. <s> BIB014 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. To the best of our knowledge, the impact of mask spoofing on face recognition has not been analyzed yet. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, we use a 2D+3D mask database which was prepared for a research project in which the authors are all involved. This paper provides new results by demonstrating the impact of mask attacks on 2D, 2.5D and 3D face recognition systems. The results show that face recognition systems are vulnerable to mask attacks, thus countermeasures have to be developed to reduce the impact of mask attacks on face recognition. The results also show that 2D texture analysis provides more information than 3D face shape analysis in order to develop a countermeasure against high-quality mask attacks. <s> BIB015 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Vulnerability to spoofing attacks is a serious drawback for many biometric systems. Among all biometric traits, face is the one that is exposed to the most serious threat, since it is exceptionally easy to access. The limited work on fraud detection capabilities for face mainly shapes around 2D attacks forged by displaying printed photos or replaying recorded videos on mobile devices. A significant portion of this work is based on the flatness of the facial surface in front of the sensor. In this study, we complicate the spoofing problem further by introducing the 3rd dimension and examine possible 3D attack instruments. A small database is constructed with six different types of 3D facial masks and experimented on to determine the right direction to study 3D attacks. Spoofing performance for each type of mask is assessed and analysed thoroughly using two Gabor-wavelet-based algorithms. <s> BIB016 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> As a crucial security problem, anti-spoofing in biometrics, and particularly for the face modality, has achieved great progress in the recent years. Still, new threats arrive inform of better, more realistic and more sophisticated spoofing attacks. The objective of the 2nd Competition on Counter Measures to 2D Face Spoofing Attacks is to challenge researchers to create counter measures effectively detecting a variety of attacks. The submitted propositions are evaluated on the Replay-Attack database and the achieved results are presented in this paper. <s> BIB017 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Face authentication is one of promising biometrics-based user authentication mechanisms that have been widely available in this era of mobile computing. With built-in camera capability on smart phones, tablets, and laptops, face authentication provides an attractive alternative of legacy passwords for its memory-less authentication process. Although it has inherent vulnerability against spoofing attacks, it is generally considered sufficiently secure as an authentication factor for common access protection. However, this belief becomes questionable since image sharing has been popular in online social networks (OSNs). A huge number of personal images are shared every day and accessible to potential adversaries. This OSN-based facial disclosure (OSNFD) creates a significant threat against face authentication. In this paper, we make the first attempt to quantitatively measure the threat of OSNFD. We examine real-world face-authentication systems designed for both smartphones, tablets, and laptops. Interestingly, our results find that the percentage of vulnerable images that can used for spoofing attacks is moderate, but the percentage of vulnerable users that are subject to spoofing attacks is high. The difference between systems designed for smartphones/tablets and laptops is also significant. In our user study, the average percentage of vulnerable users is 64% for laptop-based systems, and 93% for smartphone/tablet-based systems. This evidence suggests that face authentication may not be suitable to use as an authentication factor, as its confidentiality has been significantly compromised due to OSNFD. In order to understand more detailed characteristics of OSNFD, we further develop a risk estimation tool based on logistic regression to extract key attributes affecting the success rate of spoofing attacks. The OSN users can use this tool to calculate risk scores for their shared images so as to increase their awareness of OSNFD. <s> BIB018 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> A. FACE SPOOFING <s> Spoofing is the act of masquerading as a valid user by falsifying data to gain an illegitimate access. Vulnerability of recognition systems to spoofing attacks (presentation attacks) is still an open security issue in biometrics domain and among all biometric traits, face is exposed to the most serious threat, since it is particularly easy to access and reproduce. In this paper, many different types of face spoofing attacks have been examined and various algorithms have been proposed to detect them. Mainly focusing on 2D attacks forged by displaying printed photos or replaying recorded videos on mobile devices, a significant portion of these studies ground their arguments on the flatness of the spoofing material in front of the sensor. However, with the advancements in 3D reconstruction and printing technologies, this assumption can no longer be maintained. In this paper, we aim to inspect the spoofing potential of subject-specific 3D facial masks for different recognition systems and address the detection problem of this more complex attack type. In order to assess the spoofing performance of 3D masks against 2D, 2.5D, and 3D face recognition and to analyze various texture-based countermeasures using both 2D and 2.5D data, a parallel study with comprehensive experiments is performed on two data sets: the Morpho database which is not publicly available and the newly distributed 3D mask attack database. <s> BIB019
The use of masks or facial disguises to avoid being recognized is a practice which has been observed for centuries in the vast majority of known civilizations. Following this trend, probably the most modern version of this long-going tradition to change oneself's physical appearance, is the use of plastic surgery, which is becoming more and more popular thanks to the availability of advanced technology, its affordable cost, and the speed with which these procedures are now performed. Recently, it has been shown that, in spite of some efforts to develop specific algorithms robust to facial surgery changes BIB009 - BIB013 , the problem of recognizing a person after undergoing this type of operations is still an open challenge for automatic face authentication systems BIB003 . Even without turning to irreversible treatments, some works have also shown that face-based biometric systems may be circumvented just by wearing regular make-up BIB010 . The afore mentioned techniques (i.e., face masks, plastic surgery and make-up) are usually used to hide the user's own identity (i.e., Bob denies being Bob) and not to perform an attack in which Bob tries to impersonate John. However, it has recently been shown at a live demonstration in a biometricdedicated conference that this security scenario could change, and that methods such as surgery or make-up may be successfully used to perform direct attacks. In this conference, a female intruder was able to access a face recognition system in the place of a male user just by wearing some adequate make-up  . Apart from these still limited examples on the potential use of disguise techniques for spoofing purposes, the vast majority of face direct attacks reported in the literature may be classified in one of two groups, as shown in Fig. 4 , depending on whether the artefacts used are: i) 2D surfaces (e.g., photo, video) which are successful against 2D face recognition systems (see grey arrows in Fig. 4 ), or ii) 3D volumes (e.g., masks) which may be used to attack 2D, 2.5D and 3D face recognition technology. Such artefacts have been used to carry out three main types of attacks which present an increasing level of spoofing potential: • Photo Attacks. These fraudulent access attempts are carried out presenting to the recognition system a photograph of the genuine user. The photograph may have been taken by the attacker using a digital camera, or even retrieved from the internet after the user himself uploaded it to one of the very popular online social networks available today BIB018 . The image can then be printed on a paper (i.e., print attacks, which were the first to be systematically studied in the literature) or may be displayed on the screen of a digital device such as a mobile phone or a tablet (i.e., digital-photo attacks)  , BIB006 , BIB004 . A slightly more advanced type of photo-attack that has also been studied is the use of photographic masks. These masks are high resolution printed photographs where eyes and mouth have been cut out. At the time of the attack the impostor is placed behind  , so that certain face movements such as eye blinking are reproduced. • Video Attacks. Also referred in some cases as replay attacks. They represent a more sophisticated version of the simple photo spoofs. In this case, the attacker does not use a still image, but replays a video of the genuine client using a digital device (e.g., mobile phone, tablet or laptop) BIB011 , BIB012 . Such attacks appeared as a further step in the evolution of face spoofing and are more difficult to detect, as not only the face 2D texture is copied but also its dynamics. • Mask Attacks. In these cases the spoofing artefact is a 3D mask of the genuine client's face, increasing the difficulty to find accurate countermeasures against them. Since the complete 3D structure of the face is imitated, the use of depth cues which could be a solution to prevent the previous two types of attacks (carried out with flat surfaces), becomes inefficient against this particular threat. Although the possibility to bypass a biometric system wearing a mask imitating the face of a different user is an idea that has been circulating for some time BIB002 , these attacks are far less common than the previous two categories. Face-mask spoofing has only started to be systematically studied with the acquisition of the first mask-specific datasets BIB014 , BIB015 , which include masks of different materials and sizes BIB019 , BIB016 . In part, the scarcity of research works addressing this potential threat may be put down to the technical and economic difficulty posed by the generation of large databases of realistic masks. However, these obstacles have been significantly lessened with the recent emergence of some companies where such 3D face models may be obtained for a reasonable price. BIB001 Furthermore, self-manufacturing a face mask is also becoming more feasible and easier each day with the new generation of affordable 3D acquisition sensors, 4 dedicated scanning software  and the price decrease of 3D printing devices.  All previous attacks have a number of variants depending on the resolution of the spoofing device, the type of support used to present the fake copy (e.g., handheld or fixed support), or the external variability allowed (e.g., illumination or background conditions). These different attack versions may be found in the face spoofing databases available for research BIB017 , BIB006 , BIB004 , BIB012 , BIB014 , BIB015 , BIB007 , which are reviewed in Sect. IV-C. It is also worth highlighting that face recognition systems may also be subjected to attacks from identical twins claiming to be the same person. Strictly speaking these are not spoofing attacks (as there is no physical artifact involved) but rather zero-effort impostor attempts in which Bob presents his own biometric trait while trying to access the system as John. Although some of the anti-spoofing techniques that will be reviewed in the following sections could potentially be used against this particular vulnerability, it will not be treated in the present article. We refer the interested reader to some specific works on this topic to understand the implications and performance of face recognition systems in the presence of twins BIB008 - BIB005 .