Biometric Antispoofing Methods: A Survey in Face Recognition <s> B. ANTI-SPOOFING EVALUATION CAMPAIGNS: GENERAL ASPECTS <s> “Liveness detection”, a technique used to determine the vitality of a submitted biometric, has been implemented in fingerprint scanners in recent years. The goal for the LivDet 2011 competition is to compare software-based fingerprint liveness detection methodologies (Part 1), as well as fingerprint systems which incorporate liveness detection capabilities (Part 2), using a standardized testing protocol and large quantities of spoof and live fingerprint images. This competition was open to all academic and industrial institutions which have a solution for either software-based or system-based fingerprint vitality detection problem. Five submissions across the two parts of the competition resulted in successful completion. These submissions were: Chinese Academy of Sciences Institute of Automation (CASIA), Federico II University (Federico) and Dermalog Identification SystemsGmbH (Dermalog) for Part 1: Algorithms, and GreenBit and Dermalog for Part 2: Systems. Part 1 was evaluated using four different datasets. The best results were from Federico on the Digital Persona dataset with error for live and spoof detection of 6.2% and 11.61% respectively. The best overall results for Part 1 were Dermalog with 34.05 FerrFake and 11.825% FerrLive. Part 2 was evaluated using live subjects and spoof finger casts. The best results were from Dermalog with an error for live and spoof of 42.5% and 0.8%, respectively. <s> BIB001 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> B. ANTI-SPOOFING EVALUATION CAMPAIGNS: GENERAL ASPECTS <s> A spoof or fake is a counterfeit biometric that is used in an attempt to circumvent a biometric sensor Liveness detection distinguishes between live and fake biometric traits. Liveness detection is based on the principle that additional information can be garnered above and beyond the data procured by a standard verification system, and this additional data can be used to verify if a biometric measure is authentic. The Fingerprint Liveness Detection Competition (LivDet) goal is to compare both software-based (Part 1) and hardware-based (Part 2) fingerprint liveness detection methodologies and is open to all academic and industrial institutions. Submissions for the third edition were much more than in the previous editions of LivDet demonstrating a growing interest in the area. We had nine participants (with eleven algorithms) for Part 1 and two submissions for Part 2. <s> BIB002 </s> Biometric Antispoofing Methods: A Survey in Face Recognition <s> B. ANTI-SPOOFING EVALUATION CAMPAIGNS: GENERAL ASPECTS <s> The use of an artificial replica of a biometric characteristic in an attempt to circumvent a system is an example of a biometric presentation attack. Liveness detection is one of the proposed countermeasures, and has been widely implemented in fingerprint and iris recognition systems in recent years to reduce the consequences of spoof attacks. The goal for the Liveness Detection (LivDet) competitions is to compare software-based iris liveness detection methodologies using a standardized testing protocol and large quantities of spoof and live images. Three submissions were received for the competition Part 1; Biometric Recognition Group de Universidad Autonoma de Madrid, University of Naples Federico II, and Faculdade de Engenharia de Universidade do Porto. The best results from across all three datasets was from Federico with a rate of falsely rejected live samples of 28.6% and the rate of falsely accepted fake samples of 5.7%. <s> BIB003
In the area of anti-spoofing assessment, as in other biometricrelated scenarios  , two main types of evaluations are possible: (i) algorithm-based, also referred in the literature as technology evaluation  , thought to evaluate liveness detection modules or algorithms, independently of the rest of the system. This type of evaluation is therefore well suited to assess feature-level techniques; (ii) system-based, also known as scenario evaluation  , designed to evaluate biometric systems as a whole, including the scanner. Adequate therefore to assess sensor-level schemes where acquisition devices are specific for each system. The advantage of algorithm-based evaluations is that the same data and protocol may be used to assess all techniques. Furthermore, this benchmark can be made public, so that future software-based methods may be directly compared to the evaluation results. On the other hand, systembased evaluations are just restricted to the scope of a given competition, and no further comparison may be established with future systems, as new data would have to be acquired for each specific sensor. That is, due to their intrinsic hardware-based nature, it is not possible to acquire a single distributable database that satisfies the particular hardware acquisition requirements of each different sensor-based approach. However, it is important to highlight that it is possible to carry out competitive evaluations of complete liveness detection systems (including the acquisition sensor) and not just of independent anti-spoofing algorithms or modules. Such system-based evaluations have already started up at the fingerprint LivDet 2011 and 2013 and at the iris LivDet 2013 competitions BIB002 , BIB003 , BIB001 . In these three contests, the two evaluation modalities mentioned above (i.e., algorithm-and sensor-based) were offered to the participants: (i) submission of anti-spoofing software-based algorithms (i.e., only the liveness detection module), that were evaluated on the same prerecorded data following the same protocol (now publicly available); (ii) submission of complete functional biometric systems, that were tested on the spot performing a fixed number of real access attempts and spoofing access attempts (i.e., direct attacks), carried out with the same, or very similar artefacts to those used for the generation of the software-based database. In the latter case, a number of important factors should be taken into account in order to obtain comparable results: using the same spoofs to evaluate all systems, enrolling the same live subjects, same acquisition conditions (e.g., background, pose or illumination), controlling the possible spoofs degradation. Compared to algorithm-based evaluations, system-based ones provide a better estimation of the anti-spoofing capabilities of fully functional biometric systems, and not just of liveness detection algorithms. Such type of assessment also gives very valuable information regarding the real robustness against spoofing of commercial biometric applications which, in practice, are released as a complete finalized product and not as a group of independent modules. Furthermore, systembased evaluations represent a closer approximation to spoofing attacks that could be carried out in a real-world scenario. Another important observation worth highlighting in the field of anti-spoofing assessment, is the distribution of fake samples across datasets. Up to date, in all algorithm-based competitions that have been organised (two in face, three in fingerprint and one in iris), the train and test sets distributed to the participants contained the same type of spoofs. This means that algorithms may be trained on the same type of data that will later be used for testing. However, in a real operational scenario, algorithms have to face artefacts which are unknown to them. This way, results obtained under laboratory conditions may be an optimistic estimate of the real performance of the anti-spoofing methods being tested. This potential bias in the evaluation results between laboratory and real environments was addressed in the systems category of the LivDet 2011 and 2013 competitions BIB002 , BIB001 . In these two contests, participants did not receive any training data and were just given some general information about the three types of spoofs that would be used to attack their systems. Then, in the testing phase, in addition to these three known artefacts, two more, totally new to the systems, were also used for evaluation. A similar approach could be followed in algorithms-based assessment by limiting the diversity of fake training data compared to that used for testing.