Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Unimodal biometric systems have to contend with a variety of problems such as noisy data, intra-class variations, restricted degrees of freedom, non-universality, spoof attacks, and unacceptable error rates. Some of these limitations can be addressed by deploying multimodal biometric systems that integrate the evidence presented by multiple sources of information. This paper discusses the various scenarios that are possible in multimodal biometric systems, the levels of fusion that are plausible and the integration strategies that can be adopted to consolidate information. We also present several examples of multimodal systems that have been described in the literature. <s> BIB001 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Image set classification finds its applications in a number of real-life scenarios such as classification from surveillance videos, multi-view camera networks and personal albums. Compared with single image based classification, it offers more promises and has therefore attracted significant research attention in recent years. Unlike many existing methods which assume images of a set to lie on a certain geometric surface, this paper introduces a deep learning framework which makes no such prior assumptions and can automatically discover the underlying geometric structure. Specifically, a Template Deep Reconstruction Model (TDRM) is defined whose parameters are initialized by performing unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBMs). The initialized TDRM is then separately trained for images of each class and class-specific DRMs are learnt. Based on the minimum reconstruction errors from the learnt class-specific models, three different voting strategies are devised for classification. Extensive experiments are performed to demonstrate the efficacy of the proposed framework for the tasks of face and object recognition from image sets. Experimental results show that the proposed method consistently outperforms the existing state of the art methods. <s> BIB002 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Ear recognition is gaining on popularity in recent years. The human ear are neither affected by expressions like faces are nor do need closer touching like finger-prints do. In this paper, a novel algorithm was proposed to do ear recognition using deep convolutional neural network and provide a visualization of the learned network. We design a convolutional neural network with three convolutional layers, a fully-connected layer and a soft-max classifier. The experimental results on USTB ear database indict that our proposed method is easier botain high accuracy and outperforms the traditional method in dealing with partial occlusion. <s> BIB003 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Automatic Offline Handwritten Signature Verification has been researched over the last few decades from several perspectives, using insights from graphology, computer vision, signal processing, among others. In spite of the advancements on the field, building classifiers that can separate between genuine signatures and skilled forgeries (forgeries made targeting a particular signature) is still hard. We propose approaching the problem from a feature learning perspective. Our hypothesis is that, in the absence of a good model of the data generation process, it is better to learn the features from data, instead of using hand-crafted features that have no resemblance to the signature generation process. To this end, we use Deep Convolutional Neural Networks to learn features in a writer-independent format, and use this model to obtain a feature representation on another set of users, where we train writer-dependent classifiers. We tested our method in two datasets: GPDS-960 and Brazilian PUC-PR. Our experimental results show that the features learned in a subset of the users are discriminative for the other users, including across different datasets, reaching close to the state-of-the-art in the GPDS dataset, and improving the state-of-the-art in the Brazilian PUC-PR dataset. <s> BIB004 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> A new deep multitask learning based metric learning method is proposed.An appropriate architecture is proposed for offline signature verification.The knowledge of other signers' samples is transfered for better accuracy.The proposed method outperforms SVM and Discriminative Deep Metric Learning.It outperforms the accuracy of other published reports. Display Omitted This paper presents a novel classification method, Deep Multitask Metric Learning (DMML), for offline signature verification. Unlike existing methods that to verify questioned signatures of an individual merely consider the training samples of that class, DMML uses the knowledge from the similarities and dissimilarities between the genuine and forged samples of other classes too. To this end, using the idea of multitask and transfer learning, DMML train a distance metric for each class together with other classes simultaneously. DMML has a structure with a shared layer acting as a writer-independent approach, that is followed by separated layers which learn writer-dependent factors. We compare the proposed method against SVM, writer-dependent and writer-independent Discriminative Deep Metric Learning method on four offline signature datasets (UTSig, MCYT-75, GPDSsynthetic, and GPDS960GraySignatures) using Histogram of Oriented Gradients (HOG) and Discrete Radon Transform (DRT) features. Results of our experiments show that DMML achieves better performance compared to other methods in verifying genuine signatures, skilled and random forgeries. <s> BIB005 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Identity recognition from ear images is an active field of research within the biometric community. The ability to capture ear images from a distance and in a covert manner makes ear recognition technology an appealing choice for surveillance and security applications as well as related application domains. In contrast to other biometric modalities, where large datasets captured in uncontrolled settings are readily available, datasets of ear images are still limited in size and mostly of laboratory-like quality. As a consequence, ear recognition technology has not benefited yet from advances in deep learning and convolutionalneural networks (CNNs) and is still lacking behind other modalities that experienced significant performance gains owing to deep recognition technology. In this paper we address this problem and aim at building a CNNbased ear recognition model. We explore different strategies towards model training with limited amounts of training data and show that by selecting an appropriate model architecture, using aggressive data augmentation and selective learning on existing (pre-trained) models, we are able to learn an effective CNN·based model using a little more than 1300training images. The result of our work is the first CNN·based approach to ear recognition that is also made publicly available to the research community. With our model we are able to improve on the rank one recognition rate of the previous state-of-the-art by more than 25% on a challenging dataset of ear images captured from the web (a.k.a, in the wild). <s> BIB006 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> We propose formulations for learning features for Offline Signature Verification.A novel method that uses knowledge of forgeries from a subset of users is proposed.Learned features are used to train classifiers for other users (without forgeries).Experiments on GPDS-960 show a large improvement in state-of-the-art.Results in other 3 datasets show that the features generalize without fine-tuning. Verifying the identity of a person using handwritten signatures is challenging in the presence of skilled forgeries, where a forger has access to a persons signature and deliberately attempt to imitate it. In offline (static) signature verification, the dynamic information of the signature writing process is lost, and it is difficult to design good feature extractors that can distinguish genuine signatures and skilled forgeries. This reflects in a relatively poor performance, with verification errors around 7% in the best systems in the literature. To address both the difficulty of obtaining good features, as well as improve system performance, we propose learning the representations from signature images, in a Writer-Independent format, using Convolutional Neural Networks. In particular, we propose a novel formulation of the problem that includes knowledge of skilled forgeries from a subset of users in the feature learning process, that aims to capture visual cues that distinguish genuine signatures and forgeries regardless of the user. Extensive experiments were conducted on four datasets: GPDS, MCYT, CEDAR and Brazilian PUC-PR datasets. On GPDS-160, we obtained a large improvement in state-of-the-art performance, achieving 1.72% Equal Error Rate, compared to 6.97% in the literature. We also verified that the features generalize beyond the GPDS dataset, surpassing the state-of-the-art performance in the other datasets, without requiring the representation to be fine-tuned to each particular dataset. <s> BIB007 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Here, the authors have extensively investigated the unconstrained ear recognition problem. The authors have first shown the importance of domain adaptation, when deep convolutional neural network (CNN) models are used for ear recognition. To enable domain adaptation, the authors have collected a new ear data set using the Multi-PIE face data set, which they named as Multi-PIE ear data set. The authors have analysed in depth the effect of ear image quality, for example, illumination and aspect ratio, on the classification performance. Finally, the authors have addressed the problem of data set bias in the ear recognition field. Experiments on the UERC data set have shown that domain adaptation leads to a significant performance improvement. For example, when VGG-16 model is used and the domain adaptation is applied, an absolute increase of around 10% has been achieved. Combining different deep CNN models has further improved the accuracy by 4%. In the experiments that the authors have conducted to examine the data set bias, given an ear image, they were able to classify the data set that it has come from with 99.71% accuracy, which indicates a strong bias among the ear recognition data sets. <s> BIB008 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Convolutional neural networks (CNNs)-based deep features have been demonstrated with remarkable performance in various vision tasks, such as image classification and face verification. Compared with the hand-crafted descriptors, deep features exhibit more powerful representation ability. Typically, higher layer features contain more semantic information, while lower layer features can provide more low-level description. In addition, it turns out that the fusion of different layer features will lead to superior performance. Here, we propose a novel approach for human ear identification by combining hierarchical deep features. First, hierarchical deep features are extracted from ear images using CNN pre-trained on large-scale data set. To enhance the feature representation and reduce the high dimension of deep features, the discriminant correlation analysis (DCA) is adopted for fusing deep features from different layers for further improvement. Owing to the lack of ear images per person, the authors propose to transform the ear identification problem to the binary classification by composing pairwise samples and resolve it with the pairwise support vector machine (SVM). Experiments are conducted on four public databases: USTB I, USTB II, IIT Delhi I, and IIT Delhi II. The proposed method achieves promising recognition rate and exhibits decent performance compared with the state-of-the-art methods. <s> BIB009 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> The use of features extracted using a deep convolutional neural network (CNN) combined with a writer-dependent (WD) SVM classifier resulted in significant improvement in performance of handwritten signature verification (HSV) when compared to the previous state-of-the-art methods. In this work it is investigated whether the use of these CNN features provide good results in a writer-independent (WI) HSV context, based on the dichotomy transformation combined with the use of an SVM writer-independent classifier. The experiments performed in the Brazilian and GPDS datasets show that (i) the proposed approach outperformed other WI-HSV methods from the literature, (ii) in the global threshold scenario, the proposed approach was able to outperform the writer-dependent method with CNN features in the Brazilian dataset, (iii) in an user threshold scenario, the results are similar to those obtained by the writer-dependent method with CNN features. <s> BIB010 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Ear recognition is a popular field of research within the biometric community. It plays an important part in automatic recognition systems. The ability to capture image of the ear from a distance and perform identity recognition makes ear recognition technology an attractive choice for security application as well as other related applications. However, datasets of ear images are still limited in size, while in other biometric modal communities, like face recognition, they possess large datasets and the most of them are collected in uncontrolled condition. As a result, deep learning technology still cannot yield satisfactory result in ear recognition area. In this paper, we tackle ear recognition problem by using few-shot learning based methods. We explore different methods towards model training with limited amounts of training data and show that by using them, with the help of data augmentation, the model can be flexible and can quickly adapt to new identity to perform fast recognition. The result of our work is the first few-shot learning based work to ear recognition. With our work we are able to significantly improve the accuracy of 23% on a regular dataset, and even 21% on a challenging dataset that is collected from the web, which is comparable with state-of-the-art in ear recognition area. <s> BIB011 </s> Biometric Recognition Using Deep Learning: A Survey <s> Method <s> Our experience of the world is multimodal - we see objects, hear sounds, feel texture, smell odors, and taste flavors. Modality refers to the way in which something happens or is experienced and a research problem is characterized as multimodal when it includes multiple such modalities. In order for Artificial Intelligence to make progress in understanding the world around us, it needs to be able to interpret such multimodal signals together. Multimodal machine learning aims to build models that can process and relate information from multiple modalities. It is a vibrant multi-disciplinary field of increasing importance and with extraordinary potential. Instead of focusing on specific multimodal applications, this paper surveys the recent advances in multimodal machine learning itself and presents them in a common taxonomy. We go beyond the typical early and late fusion categorization and identify broader challenges that are faced by multimodal machine learning, namely: representation, translation, alignment, fusion, and co-learning. This new taxonomy will enable researchers to better understand the state of the field and identify directions for future research. <s> BIB012
Dataset Accuracy Zhang et al BIB011 UERC 62.48 ± 0.09% Eyiokur et al BIB008 UERC 63.62% Zhang et al BIB011 AMI 99.94 ± 0.05% Omara et al BIB009 IITD I 99.5% Omara et al BIB009 USTB II 99% Sinha et al [189] USTB III 97.9% Tian et al BIB003 USTB III 98.27% Emersic et al BIB006 Composed 62% lected in various scene and viewing conditions, while, using OU-ISIR, they are for cross-view conditions. Dataset EER Hafemann et al BIB004 GPDS-160 10.70% Yilmaz et al  GPDS-160 6.97% Souza et al BIB010 GPDS-160 2.86% Hafemann et al BIB007 GPDS-  2.63% Soleimani et al BIB005 GPDS-300 20.94% Hafemann et al BIB004 GPDS-300 12.83% Souza et al BIB010 GPDS-300 3.34% Hafemann et al BIB007 GPDS-300 3.15% guise, or after surgery may not be that easy. Fusing the information from multiple biometrics can provide a more reliable solution/system in many of these cases (for example using voice+face or voice+gait can potentially solve the identical twin detection) BIB001  . A good neural architecture which can jointly encode and aggregate different biometrics would be an interesting problem (information fusion can happen at the data level, feature level, score level, or decision level). Image set classification could also be useful in this direction BIB002 . There have been some works on biometric fusion, but most of them are far from the real-world scale of biometric recognition, and are mostly in their infancy. For some of the challenges of multi-modal machine learning, we refer the reader to BIB012 .