Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Abstract : Predicting programs designed for large general-purpose computers constitute an important new tool in the control of production and economics. Nevertheless, small predicting filters have their own domain of application. They can be realized not only as programs for general-purpose computers, but also as simple analog devices with very fast response. The authors discuss three principal methods of prediction in addition to some others. Prediction of deterministic processes, i.e. extrapolation and interpolation. Prediction of stochastic processes, based on statistical prediction theory. Prediction based on adaptation or learning of the predicting filters. (Author) <s> BIB001 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> A tutorial on the design and development of automatic speaker-recognition systems is presented. Automatic speaker recognition is the use of a machine to recognize a person from a spoken phrase. These systems can operate in two modes: to identify a particular person or to verify a person's claimed identity. Speech processing and the basic components of automatic speaker-recognition systems are shown and design tradeoffs are discussed. Then, a new automatic speaker-recognition system is given. This recognizer performs with 98.9% correct decalcification. Last, the performances of various systems are compared. <s> BIB002 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Fingerprint verification is one of the most reliable personal identification methods. However, manual fingerprint verification is incapable of meeting today's increasing performance requirements. An automatic fingerprint identification system (AFIS) is needed. This paper describes the design and implementation of an online fingerprint verification system which operates in two stages: minutia extraction and minutia matching. An improved version of the minutia extraction algorithm proposed by Ratha et al. (1995), which is much faster and more reliable, is implemented for extracting features from an input fingerprint image captured with an online inkless scanner. For minutia matching, an alignment-based elastic matching algorithm has been developed. This algorithm is capable of finding the correspondences between minutiae in the input image and the stored template without resorting to exhaustive search and has the ability of adaptively compensating for the nonlinear deformations and inexact pose transformations between fingerprints. The system has been tested on two sets of fingerprint images captured with inkless scanners. The verification accuracy is found to be acceptable. Typically, a complete fingerprint verification procedure takes, on an average, about eight seconds on a SPARC 20 workstation. These experimental results show that our system meets the response time requirements of online verification with high accuracy. <s> BIB003 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Abstract As the first attempt of automatic personal identification by palmprint, in this paper, two novel characteristics, datum point invariance and line feature matching, are presented in palmprint verification. The datum points of palmprint, which have the remarkable advantage of invariable location, are defined and their determination using the directional projection algorithm is developed. Then, line feature extraction and line matching are proposed to detect whether a couple of palmprints are from the same palm. Various palmprint images have been tested to illustrate the effectiveness of the palmprint verification with both characteristics. <s> BIB004 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> More than ever before the Internet is changing computing as we know it. Global access to information and resources is becoming an integral part of nearly every aspect of our lives. Unfortunately, with this global network access comes increased chances of malicious attack and intrusion. In an effort to confront the new threats unveiled by the networking revolution of the past few years reliable, rapid, and unintrusive means for automatically recognizing the identity of individuals are now being sought. In this paper we examine an emerging non-static biometric technique that aims to identify users based on analyzing habitual rhythm patterns in the way they type. ©2000 Elsevier Science B.V. All rights reserved. <s> BIB005 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Biometrics-based authentication and identification are emerging as the most reliable method to authenticate and identify individuals. Biometrics requires that the person to be identified be physically present at the point-of-identification and relies on `something which you are or you do' to provide better security, increased efficiency, and improved accuracy. Automated biometrics deals with physiological or behavioral characteristics such as fingerprints, signature, palmprint, iris, hand, voice and face that can be used to authenticate a person's identity or establish an identity from a database. With rapid progress in electronic and Internet commerce, there is also a growing need to authenticate the identity of a person for secure transaction processing. Designing an automated biometrics system to handle large population identification, accuracy and reliability of authentication are challenging tasks. Currently, there are over ten different biometrics systems that are either widely used or under development. Some automated biometrics, such as fingerprint identification and speaker verification, have received considerable attention over the past 25 years, and some issues like face recognition and iris-based authentication have been studied extensively resulting in successful development of biometrics systems in commercial applications. However, very few books are exclusively devoted to such issues of automated biometrics. Automated Biometrics: Technologies and Systems systematically introduces the technologies and systems, and explores how to design the corresponding systems with in-depth discussion. The issues addressed in this book are highly relevant to many fundamental concerns of both researchers and practitioners of automated biometrics in computer and system security. <s> BIB006 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> The principle that underlies the recognition of persons by their iris patterns is the failure of a test of statistical independence on texture phase structure as encoded by multiscale quadrature wavelets. The combinatorial complexity of this phase information across different persons spans about 249 degrees of freedom and generates a discrimination entropy of about 3.2 bits/mm/sup 2/ over the iris, enabling real-time decisions about personal identity with extremely high confidence. Algorithms first described by the author in 1993 have now been tested in several independent field trials and are becoming widely licensed. This presentation reviews how the algorithms work and presents the results of 9.1 million comparisons among different eye images acquired in trials in Britain, the USA, Korea, and Japan. <s> BIB007 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Human identification at a distance has recently gained growing interest from computer vision researchers. Gait recognition aims essentially to address this problem by identifying people based on the way they walk. In this paper, a simple but efficient gait recognition algorithm using spatial-temporal silhouette analysis is proposed. For each image sequence, a background subtraction algorithm and a simple correspondence procedure are first used to segment and track the moving silhouettes of a walking figure. Then, eigenspace transformation based on principal component analysis (PCA) is applied to time-varying distance signals derived from a sequence of silhouette images to reduce the dimensionality of the input feature space. Supervised pattern classification techniques are finally performed in the lower-dimensional eigenspace for recognition. This method implicitly captures the structural and transitional characteristics of gait. Extensive experimental results on outdoor image sequences demonstrate that the proposed algorithm has an encouraging recognition performance with relatively low computational cost. <s> BIB008 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> As one of the most successful applications of image analysis and understanding, face recognition has recently received significant attention, especially during the past several years. At least two reasons account for this trend: the first is the wide range of commercial and law enforcement applications, and the second is the availability of feasible technologies after 30 years of research. Even though current machine recognition systems have reached a certain level of maturity, their success is limited by the conditions imposed by many real applications. For example, recognition of face images acquired in an outdoor environment with changes in illumination and/or pose remains a largely unsolved problem. In other words, current systems are still far away from the capability of the human perception system.This paper provides an up-to-date critical survey of still- and video-based face recognition research. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the studies of machine recognition of faces. To provide a comprehensive survey, we not only categorize existing recognition techniques but also present detailed descriptions of representative methods within each category. In addition, relevant topics such as psychophysical studies, system evaluation, and issues of illumination and pose variation are covered. <s> BIB009 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Biometric identication is an emerging technology that can solve security problems in our networked society. A few years ago, a new branch of biometric technology, palmprint authentication, was proposed (Pattern Recognition 32(4) (1999) 691) whereby lines and points are extracted from palms for personal identication. In this paper, we consider the palmprint as a piece of texture and apply texture-based feature extraction techniques to palmprint authentication. A 2-D Gaborlter is used to obtain texture information and two palmprint images are compared in terms of their hamming distance. The experimental results illustrate the eectiveness of our method. ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. <s> BIB010 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Application and research of ear recognition technology is a new subject in the field of biometric recognition The earlier research has shown that human ear is one of the representative human biometrics with uniqueness and stability The paper discusses the edge-based ear recognition method including ear edge detection, ear description and feature extraction, recognition method and ear database construction The feature vector is composed of the shape feature vector of the outer ear and the structural feature vector of the inner ear The local feature vectors are proved to be invariant to ear image's parallel move, scale and rotation. <s> BIB011 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Human identification at a distance has recently gained growing interest from computer vision researchers. This paper aims to propose a visual recognition algorithm based upon fusion of static and dynamic body biometrics. For each sequence involving a walking figure, pose changes of the segmented moving silhouettes are represented as an associated sequence of complex vector configurations, and are then analyzed using the Procrustes shape analysis method to obtain a compact appearance representation, called static information of body. Also, a model-based approach is presented under a condensation framework to track the walker and to recover joint-angle trajectories of lower limbs, called dynamic information of gait. Both static and dynamic cues are respectively used for recognition using the nearest exemplar classifier. They are also effectively fused on decision level using different combination rules to improve the performance of both identification and verification. Experimental results on a dataset including 20 subjects demonstrate the validity of the proposed algorithm. <s> BIB012 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> In this work, we present a novel approach to face recognition which considers both shape and texture information to represent face images. The face area is first divided into small regions from which Local Binary Pattern (LBP) histograms are extracted and concatenated into a single, spatially enhanced feature histogram efficiently representing the face image. The recognition is performed using a nearest neighbour classifier in the computed feature space with Chi square as a dissimilarity measure. Extensive experiments clearly show the superiority of the proposed scheme over all considered methods (PCA, Bayesian Intra/extrapersonal Classifier and Elastic Bunch Graph Matching) on FERET tests which include testing the robustness of the method against different facial expressions, lighting and aging of the subjects. In addition to its efficiency, the simplicity of the proposed method allows for very fast feature extraction. <s> BIB013 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> In this paper, a new technique coined two-dimensional principal component analysis (2DPCA) is developed for image representation. As opposed to PCA, 2DPCA is based on 2D image matrices rather than 1D vectors so the image matrix does not need to be transformed into a vector prior to feature extraction. Instead, an image covariance matrix is constructed directly using the original image matrices, and its eigenvectors are derived for image feature extraction. To test 2DPCA and evaluate its performance, a series of experiments were performed on three face image databases: ORL, AR, and Yale face databases. The recognition rate across all trials was higher using 2DPCA than PCA. The experimental results also indicated that the extraction of image features is computationally more efficient using 2DPCA than PCA. <s> BIB014 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds. <s> BIB015 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> This work reports a preliminary study on the vulnerability evaluation of fingerprint verification systems to direct attacks carried out with fake fingertips created from minutiae templates. The attack is performed by presenting to the acquisition sensor a fake fingertip generated from an image reconstructed from a compromised minutiae-based template. Experiments carried out against a state-of-the-art fingerprint recognition algorithm show that the proposed attack scheme is definitely feasible and highlight a potential security threat in the use of non-encrypted minutiae-based templates. <s> BIB016 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> While retina recognition is recognized as a highly accurate and difficult to forge biometric, it has not seen widespread acceptance. In addition to user acceptance of what is at times considered an invasive technique, this limited acceptance was caused in part by the relatively high cost of signal acquisition. To alleviate the latter concern, this paper therefore describes a retina recognition algorithm based on Hill's algorithm together with a description of its implementation and experimental evaluation. The algorithm is designed to operate with signals acquired in the visual spectrum, which can be obtained using a number of off-the-shelf camera systems and provides acceptable performance compared to approaches based on near-infrared retinal images. <s> BIB017 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> I. Fingerprint Analysis The History of Fingerprints Outline of the History of Fingerprints Early Nonsystematic Methods of Identification Study Questions Systematic Methods of Identification Bertillonage Fingerprints Definitions Associated with Fingerprints Fingerprint Patterns and Ridge Characteristics More Fingerprint Definitions Study Questions Fingerprint Pattern Types and AssociatedTerminology Loop Pattern Loop Ridge Counting Radial and Ulnar Loops Other Issues Pertaining to Loops Plain Arch and Tented Arch Patterns The Plain Arch The Tented Arch The Whorl Pattern Plain Whorl Central Pocket Loop Whorl Double Loop Whorl Accidental Whorl Ridge Tracing and Counting Whorl Patterns The Palm Print Study Questions Classification: Henry with FBI Extension, NCIC, and IAFIS Henry With FBI Extension Ridge Count Conversion Conversion Chart for Ridge Counts Referencing Filing Sequence NCIC Classification System IAFIS Other Fingerprint Systems (Manual Methods or Systems) Study Questions II. Development, Identification, and Presentation of Fingerprints Known/Direct/Inked Fingerprints Unknown/ Latent Fingerprints Known Fingerprints Manual Method Live Scan Unknown Fingerprints (Commonly Referred to as Latent Prints) Conditions Affecting Latent Prints The Crime Scene Search and Fingerprint Development Fingerprint Development Techniques (see Appendix A) Powder Techniques Chemical Techniques Procedure After Developing Prints Fingerprint Comparison and Identification Study Questions Court Presentation Preparation Court Presentation Verbal Nonverbal Review Challenges to the Science of Fingerprints Study Questions Appendix A Appendix B Bibliography Index <s> BIB018 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> With the increasing concerns on security breaches and transaction fraud, highly reliable and convenient personal verification and identification technologies are more and more requisite in our social activities and national services. Biometrics, used to recognize the identity of an individual, are gaining ever-growing popularity in an extensive array of governmental, military, forensic, and commercial security applications. Advanced Pattern Recognition Technologies with Applications to Biometrics focuses on two kinds of advanced biometric recognition technologies, biometric data discrimination and multi-biometrics, while systematically introducing recent research in developing effective biometric recognition technologies. Organized into three main sections, this cutting-edge book explores advanced biometric data discrimination technologies, describes tensor-based biometric data discrimination technologies, and develops the fundamental conception and categories of multi-biometrics technologies. <s> BIB019 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Human identification by gait has created a great deal of interest in computer vision community due to its advantage of inconspicuous recognition at a relatively far distance. This paper provides a comprehensive survey of recent developments on gait recognition approaches. The survey emphasizes on three major issues involved in a general gait recognition system, namely gait image representation, feature dimensionality reduction and gait classification. Also, a review of the available public gait datasets is presented. The concluding discussions outline a number of research challenges and provide promising future directions for the field. <s> BIB020 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Principal component analysis PCA is a multivariate technique that analyzes a data table in which observations are described by several inter-correlated quantitative dependent variables. Its goal is to extract the important information from the table, to represent it as a set of new orthogonal variables called principal components, and to display the pattern of similarity of the observations and of the variables as points in maps. The quality of the PCA model can be evaluated using cross-validation techniques such as the bootstrap and the jackknife. PCA can be generalized as correspondence analysis CA in order to handle qualitative variables and as multiple factor analysis MFA in order to handle heterogeneous sets of variables. Mathematically, PCA depends upon the eigen-decomposition of positive semi-definite matrices and upon the singular value decomposition SVD of rectangular matrices. Copyright © 2010 John Wiley & Sons, Inc. <s> BIB021 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> The personal identification approaches using iris images are receiving increasing attention in the biometrics literature. Several methods have been presented in the literature and those based on the phase encoding of texture information are suggested to be the most promising. However, there has not been any attempt to combine these approaches to achieve further improvement in the performance. This paper presents a comparative study of the performance from the iris authentication using Log-Gabor, Haar wavelet, DCT and FFT based features. Our experimental results suggest that the performance from the Haar wavelet and Log-Gabor filter based phase encoding is the most promising among all the four approaches considered in this work. Therefore, the combination of these two matchers is most promising, both in terms of performance and the computational complexity. Our experimental results from the all 411 users (CASIA v3) and 224 users (IITD v1) database illustrate significant improvement in the performance which is not possible with either of these approaches individually. <s> BIB022 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> This highly anticipated new edition provides a comprehensive account of face recognition research and technology, spanning the full range of topics needed for designing operational face recognition systems. After a thorough introductory chapter, each of the following chapters focus on a specific topic, reviewing background information, up-to-date techniques, and recent results, as well as offering challenges and future directions. Features: fully updated, revised and expanded, covering the entire spectrum of concepts, methods, and algorithms for automated face detection and recognition systems; provides comprehensive coverage of face detection, tracking, alignment, feature extraction, and recognition technologies, and issues in evaluation, systems, security, and applications; contains numerous step-by-step algorithms; describes a broad range of applications; presents contributions from an international selection of experts; integrates numerous supporting graphs, tables, charts, and performance data. <s> BIB023 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade. <s> BIB024 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overriding in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. <s> BIB025 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. <s> BIB026 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Describing several new biometric technologies, such as high-resolution fingerprint, finger-knuckle-print, multi-spectral backhand, 3D fingerprint, tongueprint, 3D ear, and multi-spectral iris recognition technologies, this book analyzes a number of efficient feature extraction, matching and fusion algorithms and how potential systems have been developed. Focusing on how to develop new biometric technologies based on the requirements of applications, and how to design efficient algorithms to deliver better performance, the work is based on the authors research with experimental results under different challenging conditions described in the text. The book offers a valuable resource for researchers, professionals and postgraduate students working in the fields of computer vision, pattern recognition, biometrics, and security applications, amongst others. <s> BIB027 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> On-line signature verification still remains a challenging task within biometrics. Due to their behavioural nature (opposed to anatomic biometric traits), signatures present a notable variability even between successive realizations. This leads to higher error rates than other largely used modalities such as iris or fingerprints and is one of the main reasons for the relatively slow deployment of this technology. As a step towards the improvement of signature recognition accuracy, the present paper explores and evaluates a novel approach that takes advantage of the performance boost that can be reached through the fusion of on-line and off-line signatures. In order to exploit the complementarity of the two modalities, we propose a method for the generation of enhanced synthetic static samples from on-line data. Such synthetic off-line signatures are used on a new on-line signature recognition architecture based on the combination of both types of data: real on-line samples and artificial off-line signatures synthesized from the real data. The new on-line recognition approach is evaluated on a public benchmark containing both real versions (on-line and off-line) of the exactly same signatures. Different findings and conclusions are drawn regarding the discriminative power of on-line and off-line signatures and of their potential combination both in the random and skilled impostors scenarios. HighlightsNovel generation method of dynamically enhanced synthetic off-line signatures.Novel on-line signature verification architecture.Findings on the fusion of on-line and off-line signature.Findings on the differences between the random and skilled forgeries scenarios.Fully reproducible protocol carried out on a large public benchmark. <s> BIB028 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> This paper presents a local feature based shape matching algorithm for expression-invariant 3D face recognition. Each 3D face is first automatically detected from a raw 3D data and normalized to achieve pose invariance. The 3D face is then represented by a set of keypoints and their associated local feature descriptors to achieve robustness to expression variations. During face recognition, a probe face is compared against each gallery face using both local feature matching and 3D point cloud registration. The number of feature matches, the average distance of matched features, and the number of closest point pairs after registration are used to measure the similarity between two 3D faces. These similarity metrics are then fused to obtain the final results. The proposed algorithm has been tested on the FRGC v2 benchmark and a high recognition performance has been achieved. It obtained the state-of-the-art results by achieving an overall rank-1 identification rate of 97.0% and an average verification rate of 99.01% at 0.001 false acceptance rate for all faces with neutral and non-neutral expressions. Further, the robustness of our algorithm under different occlusions has been demonstrated on the Bosphorus dataset. <s> BIB029 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Abstract Iris recognition is one of the most promising fields in biometrics. Notwithstanding this, there are not so many research works addressing it by machine learning techniques. In this survey, we especially focus on recognition, and leave the detection and feature extraction problems in the background. However, the kind of features used to code the iris pattern may significantly influence the complexity of the methods and their performance. In other words, complexity affects learning, and iris patterns require relatively complex feature vectors, even if their size can be optimized. A cross-comparison of these two parameters, feature complexity vs. learning effectiveness, in the context of different learning algorithms, would require an unbiased common benchmark. Moreover, at present it is still very difficult to reproduce techniques and experiments due to the lack of either sufficient implementation details or reliable shared code. <s> BIB030 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Automatic identity recognition from ear images represents an active field of research within the biometric community. The ability to capture ear images from a distance and in a covert manner makes the technology an appealing choice for surveillance and security applications as well as other application domains. Significant contributions have been made in the field over recent years, but open research problems still remain and hinder a wider (commercial) deployment of the technology. This paper presents an overview of the field of automatic ear recognition (from 2D images) and focuses specifically on the most recent, descriptor-based methods proposed in this area. Open challenges are discussed and potential research directions are outlined with the goal of providing the reader with a point of reference for issues worth examining in the future. In addition to a comprehensive review on ear recognition technology, the paper also introduces a new, fully unconstrained dataset of ear images gathered from the web and a toolbox implementing several state-of-the-art techniques for ear recognition. The dataset and toolbox are meant to address some of the open issues in the field and are made publicly available to the research community. <s> BIB031 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> This chapter introduces an offline signature recognition technique using rough neural network and rough set. Rough neural network tries to find better recognition performance to classify the input offline signature images. Rough sets have provided an array of tools which turned out to be especially adequate for conceptualization, organization, classification, and analysis of various types of data, when dealing with inexact, uncertain, or vague knowledge. Also, rough sets discover hidden pattern and regularities in application. This new hybrid technique achieves good results, since the short rough neural network algorithm is neglected by the grid features technique, and then the advantages of both techniques are integrated. <s> BIB032 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> The presence of a corresponding talking face has been shown to significantly improve speech intelligibility in noisy conditions and for hearing impaired population. In this paper, we present a system that can generate landmark points of a talking face from an acoustic speech in real time. The system uses a long short-term memory (LSTM) network and is trained on frontal videos of 27 different speakers with automatically extracted face landmarks. After training, it can produce talking face landmarks from the acoustic speech of unseen speakers and utterances. The training phase contains three key steps. We first transform landmarks of the first video frame to pin the two eye points into two predefined locations and apply the same transformation on all of the following video frames. We then remove the identity information by transforming the landmarks into a mean face shape across the entire training dataset. Finally, we train an LSTM network that takes the first- and second-order temporal differences of the log-mel spectrogram as input to predict face landmarks in each frame. We evaluate our system using the mean-squared error (MSE) loss of landmarks of lips between predicted and ground-truth landmarks as well as their first- and second-order temporal differences. We further evaluate our system by conducting subjective tests, where the subjects try to distinguish the real and fake videos of talking face landmarks. Both tests show promising results. <s> BIB033 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> In this work, we describe a new deep learning based method that can effectively distinguish AI-generated fake videos (referred to as {\em DeepFake} videos hereafter) from real videos. Our method is based on the observations that current DeepFake algorithm can only generate images of limited resolutions, which need to be further warped to match the original faces in the source video. Such transforms leave distinctive artifacts in the resulting DeepFake videos, and we show that they can be effectively captured by convolutional neural networks (CNNs). Compared to previous methods which use a large amount of real and DeepFake generated images to train CNN classifier, our method does not need DeepFake generated images as negative training examples since we target the artifacts in affine face warping as the distinctive feature to distinguish real and fake images. The advantages of our method are two-fold: (1) Such artifacts can be simulated directly using simple image processing operations on a image to make it as negative example. Since training a DeepFake model to generate negative examples is time-consuming and resource-demanding, our method saves a plenty of time and resources in training data collection; (2) Since such artifacts are general existed in DeepFake videos from different sources, our method is more robust compared to others. Our method is evaluated on two sets of DeepFake video datasets for its effectiveness in practice. <s> BIB034 </s> Biometric Recognition Using Deep Learning: A Survey <s> Introduction <s> Our experience of the world is multimodal - we see objects, hear sounds, feel texture, smell odors, and taste flavors. Modality refers to the way in which something happens or is experienced and a research problem is characterized as multimodal when it includes multiple such modalities. In order for Artificial Intelligence to make progress in understanding the world around us, it needs to be able to interpret such multimodal signals together. Multimodal machine learning aims to build models that can process and relate information from multiple modalities. It is a vibrant multi-disciplinary field of increasing importance and with extraordinary potential. Instead of focusing on specific multimodal applications, this paper surveys the recent advances in multimodal machine learning itself and presents them in a common taxonomy. We go beyond the typical early and late fusion categorization and identify broader challenges that are faced by multimodal machine learning, namely: representation, translation, alignment, fusion, and co-learning. This new taxonomy will enable researchers to better understand the state of the field and identify directions for future research. <s> BIB035
Biometric features 1 hold a unique place when it comes to recognition, authentication, and security applications  , BIB006 . They cannot get lost, unlike tokenbased features such as keys and ID cards, and they cannot be forgotten, unlike knowledge-based features, such as passwords or answers to security questions  . In addition, they are almost impossible to perfectly imitate or duplicate. Even though there have been recent attempts to generate and forge various biometric features BIB016 , BIB033 , there have also been methods proposed to distinguish fake biometric features from authentic ones [6],  , BIB034 . Changes over time for many biometric features are also extremely little. For these reasons, they have been utilized in many applications, including cellphone authentication, airport security, and forensic science. Biometric features can be physiological, which are features possessed by any person, such as fingerprints  , palmprints  , BIB004 , facial features BIB009 , ears BIB011 , irises BIB007 ,  , and retinas BIB017 , or behavioral, which are apparent in a person's interaction with the environment, such as signatures BIB032 , gaits BIB020 , and keystroke BIB005 . Voice/Speech contains both behavioral features, such as accent, and physiological features, such as voice pitch BIB002 . Face and fingerprint are arguably the most commonly used physiological biometric feature. Fingerprint is the oldest, dating back to 1893 when it was used to convict a murder suspect in Argentina BIB018 . Face has many discriminative features which can be used for recognition tasks BIB023 . However, its susceptibility to change due to factors such as expression or aging, may present a challenge BIB029 ,  . Fingerprint consists of ridges and valleys, which form unique patterns. Minutiae are major local portions of the fingerprint which can be used to determine the uniqueness of the fingerprint, two of the most important ones of which being ridge endings and ridge bifurcations BIB003 . Palmprint is another alternative used for authentication purposes. In addition to minutiae features, palmprints also consist of geometry-based features, delta points, principal lines, and wrinkles BIB027 . Iris and retina are the two most popular biometrics that are present in the eye, and can be used for recognition through the texture of the iris or the pattern of blood vessels in the retina. One interesting point worth noting is that even the two eyes in the same person have different patterns BIB030 . Ears can also be used as a biometric through the shape of their lobes, and helix, and unlike most biometric features, do not need the person's direct interaction. The right and left ears are symmetrical in a person in most cases. However, their sizes are subject to change over time BIB031 . Among the behavioral features, signatures are arguably the most widely used today. The strokes in the signature can be examined for the pressure of the pen throughout the signature as well as the speed, which is a factor in the thickness of the stroke BIB028 . Gait refers to the manner of walking, which has been gaining more attention in the recent years. Due to the involvement of many joints and body parts in the process of walking, gait can also be used to uniquely identify a person from a distance BIB012 . Samples of various biometrics are shown in Figure 1 . Traditionally, the biometric recognition process involved several key steps. Figure 2 shows the blockdiagram of traditional biometric recognition systems. Firstly, the image data are acquired via (various) camera or optical sensors, and are then pre-processed so as to make the algorithm work on as much useful data as possible. Then, features are extracted from each image. Classical biometric recognition works were mostly based on hand-crafted features (designed by computer vision experts) to work with a certain type of data BIB013 ,  , BIB019 . Many of the hand-crafted features were based on the distribution of edges (SIFT BIB024 , HOG BIB015 ), or where derived from transform domain, such as Gabor BIB010 , Fourier  , and wavelet  . Principal component analysis is also used in many works to reduce the dimensionality of the features BIB021 , BIB014 . Once the features are extracted, they are fed into a classifier to perform recognition. The images in the first to sixth rows denote samples from face, fingerprint, iris, palmprint, ear, and gait respectively BIB008 [32]  [34] BIB022  . Many challenges arise in a traditional biometric recognition task. For example, the hand-crafted features that are suitable for one biometric, will not necessarily perform well on others. Therefore, it would take a great number of experiments to find and choose the most efficient set of hand-crafted features for a certain biometric. Also many of the classical models were based on multi-class SVM trained in an one-vs-one fashion, which will not scale well when the number of classes is large. However, a paradigm shift started to occur in 2012, when a deep learning-based model, AlexNet BIB025 , won the ImageNet competition by a large margin. Since then, deep learning models have been applied to a wide range of problems in computer vision and Natural Language Processing (NLP), and achieved promising results. Not surprisingly, biometric recognition methods were not an exception, and were taken over by deep learning models (with a few years delay). Deep learning based models provide an end-to-end learning framework, which can jointly learn the feature representation while performing classification/regression. This is achieved through a multi-layer neural networks, also known as Deep Neural Networks (DNNs), to learn multiple levels of representations that correspond to different levels of abstraction, which is better suited to uncover underlying patterns of the data (as shown in Figure 3 ). The idea of a multi-layer neural network dates back to the 1960s BIB001 BIB026 . However, their feasible implementation was a challenge in itself, as the training time would be too large (due to lack of powerful computers at that time). The progresses made in processor technology, and especially the development of General-Purpose GPUs (GPGPUs), as well as development of new techniques (such as Dropout) for training neural networks with a lower chance of over-fitting, enabled scientists to train very deep neural networks much faster  . The main idea of a neural network is to pass the (raw) data through a series of interconnected neurons or nodes, each of which emulates a linear or non-linear function based on its own weights and biases. These weights and biases would change during the training through backpropagation of the gradients from the output  , usually resulted from the differences between the expected output and the actual current output, aimed to minimized a loss function or cost function (difference between the predicted and actual outputs according to some metric)  . We will talk about different deep architectures in more details in Section 2. Using deep models for biometric recognition, one can learn a hierarchy of concepts as we go deeper in the network. Looking at face recognition for example, as shown in Figure 3 , starting from the first few layers of the deep neural network, we can observe learned patterns similar to the Gabor feature (oriented edges with different scales). The next few layers can learn more complex texture features and part of the face. The following layers are able to catch more complex pattern, such as high-bridged nose and big eyes. Finally the last few layers can learn very abstract concepts and certain facial attribute (such as smile, roar, and even eye color faces). In this paper, we present a comprehensive review of the recent advances in biometric recognition using deep learning frameworks. For each work, we provide an overview of the the key contributions, network architecture, and loss functions, developed to push state-ofthe-art performance in biometric recognition. We have gathered more than 150 papers, which appeared between 2014 and 2019, in leading computer vision, biometric recognition, and machine learning conferences and journals. For each biometric, we provide some of the most popular datasets used by the computer vision community, and the most promising state-of-theart deep learning works utilized in the area of biometric recognition. We then provide a quantitative analysis of well-known models for each biometric. Finally, we explore the challenges associated with deep learningbased methods in biometric recognition and research opportunities for the future. The goal of this survey is to help new researchers in this field to navigate through the progress of deep learning-based biometric recognition models, particularly with the growing interest of multi-modal biometrics systems BIB035 . Compared to the existing literature, the main contributions of this paper are as follow: -To the best of our knowledge, this is the only review paper which provides an overview of eight popular biometrics proposed before and in 2019, including face, fingerprint, iris, palmprint, ear, voice, signature, and gait. -We cover the contemporary literature with respect to this area. We present a comprehensive review of more than 150 methods, which have appeared since 2014. -We provide a comprehensive review and an insightful analysis of different aspects of biometric recognition using deep learning, including the training data, the choice of network architectures, training strategies, and their key contributions. -We provide a comparative summary of the properties and performance of the reviewed methods for biometric recognition. -We provide seven challenges and potential future direction for deep learning-based biometric recognition models. The structure of the rest of this paper is as follows. In Section 2, we provide an overview of popular deep neural networks architectures, which serve as the backbone of many biometric recognition algorithms, including convolutional neural networks, recurrent neural networks, auto-encoders, and generative adversarial networks. Then in Section 3, we provide an introduction to each of the eight biometrics (Face, Fingerprint, Iris, Palmprint, Ear, Voice, Signature, and Gait), some of the popular datasets for each of them, as well as the promising deep learning based works developed for them. The quantitative results and experimental performance of these models for all biometrics are provided in Section 4. Finally in Section 5, we explore the challenges and future directions for deep learning-based biometric recognition. 