Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Ear Recognition <s> Information fusion is a key step in multimodal biometric systems. The fusion of information can occur at different levels of a recognition system, i.e., at the feature level, matching-score level, or decision level. However, feature level fusion is believed to be more effective owing to the fact that a feature set contains richer information about the input biometric data than the matching score or the output decision of a classifier. The goal of feature fusion for recognition is to combine relevant information from two or more feature vectors into a single one with more discriminative power than any of the input feature vectors. In pattern recognition problems, we are also interested in separating the classes. In this paper, we present discriminant correlation analysis (DCA), a feature level fusion technique that incorporates the class associations into the correlation analysis of the feature sets. DCA performs an effective feature fusion by maximizing the pairwise correlations across the two feature sets and, at the same time, eliminating the between-class correlations and restricting the correlations to be within the classes. Our proposed method can be used in pattern recognition applications for fusing the features extracted from multiple modalities or combining different feature vectors extracted from a single modality. It is noteworthy that DCA is the first technique that considers class structure in feature fusion. Moreover, it has a very low computational complexity and it can be employed in real-time applications. Multiple sets of experiments performed on various biometric databases and using different feature extraction techniques, show the effectiveness of our proposed method, which outperforms other state-of-the-art approaches. <s> BIB001 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Ear Recognition <s> In this paper we present the results of the Unconstrained Ear Recognition Challenge (UERC), a group benchmarking effort centered around the problem of person recognition from ear images captured in uncontrolled conditions. The goal of the challenge was to assess the performance of existing ear recognition techniques on a challenging large-scale dataset and identify open problems that need to be addressed in the future. Five groups from three continents participated in the challenge and contributed six ear recognition techniques for the evaluation, while multiple baselines were made available for the challenge by the UERC organizers. A comprehensive analysis was conducted with all participating approaches addressing essential research questions pertaining to the sensitivity of the technology to head rotation, flipping, gallery size, large-scale recognition and others. The top performer of the UERC was found to ensure robust performance on a smaller part of the dataset (with 180 subjects) regardless of image characteristics, but still exhibited a significant performance drop when the entire dataset comprising 3,704 subjects was used for testing. <s> BIB002 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Ear Recognition <s> Identity recognition from ear images is an active field of research within the biometric community. The ability to capture ear images from a distance and in a covert manner makes ear recognition technology an appealing choice for surveillance and security applications as well as related application domains. In contrast to other biometric modalities, where large datasets captured in uncontrolled settings are readily available, datasets of ear images are still limited in size and mostly of laboratory-like quality. As a consequence, ear recognition technology has not benefited yet from advances in deep learning and convolutionalneural networks (CNNs) and is still lacking behind other modalities that experienced significant performance gains owing to deep recognition technology. In this paper we address this problem and aim at building a CNNbased ear recognition model. We explore different strategies towards model training with limited amounts of training data and show that by selecting an appropriate model architecture, using aggressive data augmentation and selective learning on existing (pre-trained) models, we are able to learn an effective CNN·based model using a little more than 1300training images. The result of our work is the first CNN·based approach to ear recognition that is also made publicly available to the research community. With our model we are able to improve on the rank one recognition rate of the previous state-of-the-art by more than 25% on a challenging dataset of ear images captured from the web (a.k.a, in the wild). <s> BIB003 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Ear Recognition <s> The authors perform unconstrained ear recognition using transfer learning with deep neural networks (DNNs). First, they show how existing DNNs can be used as a feature extractor. The extracted features are used by a shallow classifier to perform ear recognition. Performance can be improved by augmenting the training dataset with small image transformations. Next, they compare the performance of the feature-extraction models with fine-tuned networks. However, because the datasets are limited in size, a fine-tuned network tends to over-fit. They propose a deep learning-based averaging ensemble to reduce the effect of over-fitting. Performance results are provided on unconstrained ear recognition datasets, the AWE and CVLE datasets as well as a combined AWE + CVLE dataset. They show that their ensemble results in the best recognition performance on these datasets as compared to DNN feature-extraction based models and single fine-tuned models. <s> BIB004 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Ear Recognition <s> Convolutional neural networks (CNNs)-based deep features have been demonstrated with remarkable performance in various vision tasks, such as image classification and face verification. Compared with the hand-crafted descriptors, deep features exhibit more powerful representation ability. Typically, higher layer features contain more semantic information, while lower layer features can provide more low-level description. In addition, it turns out that the fusion of different layer features will lead to superior performance. Here, we propose a novel approach for human ear identification by combining hierarchical deep features. First, hierarchical deep features are extracted from ear images using CNN pre-trained on large-scale data set. To enhance the feature representation and reduce the high dimension of deep features, the discriminant correlation analysis (DCA) is adopted for fusing deep features from different layers for further improvement. Owing to the lack of ear images per person, the authors propose to transform the ear identification problem to the binary classification by composing pairwise samples and resolve it with the pairwise support vector machine (SVM). Experiments are conducted on four public databases: USTB I, USTB II, IIT Delhi I, and IIT Delhi II. The proposed method achieves promising recognition rate and exhibits decent performance compared with the state-of-the-art methods. <s> BIB005 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Ear Recognition <s> Ear recognition is a popular field of research within the biometric community. It plays an important part in automatic recognition systems. The ability to capture image of the ear from a distance and perform identity recognition makes ear recognition technology an attractive choice for security application as well as other related applications. However, datasets of ear images are still limited in size, while in other biometric modal communities, like face recognition, they possess large datasets and the most of them are collected in uncontrolled condition. As a result, deep learning technology still cannot yield satisfactory result in ear recognition area. In this paper, we tackle ear recognition problem by using few-shot learning based methods. We explore different methods towards model training with limited amounts of training data and show that by using them, with the help of data augmentation, the model can be flexible and can quickly adapt to new identity to perform fast recognition. The result of our work is the first few-shot learning based work to ear recognition. With our work we are able to significantly improve the accuracy of 23% on a regular dataset, and even 21% on a challenging dataset that is collected from the web, which is comparable with state-of-the-art in ear recognition area. <s> BIB006
Ear recognition is not as popular as face, iris, and fingerprint recognition yet. Therefore, datasets used for this procedure are still limited in size. Based on this, Zhang et al BIB006 proposed few-shot learning methods, where the network use the limited training and quickly learn to recognize the images. Dodge et al BIB004 , who proposed using transfer learning with deep networks for unconstrained ear recognition Emersic and colleagues BIB002 , also proposed a deep learning-based averaging system to mitigate the overfitting caused by the small size of the datasets. In BIB003 , the authors proposed the first publicly available CNN-based ear recognition method. They explored different strategies, such as different architectures, selective learning on pre-trained data and aggressive data augmentation to find the best configurations for their work. In  , the authors showed how ear accessories can disrupt the recognition process and even be used for spoofing, especially in a CNN-based method, e.g., VGG-16, against a traditional method, e.g., local binary patterns (LBP), and proposed methods to remove such accessories and improve the performance, such as "inprinting" and area coloring. Sinha et al [189] proposed a framework which localizes the outer ear image using HOG and SVMs, and then uses CNNs to perform ear recognition. It aims to resolve the issues usually associated with feature extraction appearance-based techniques, namely the conditions in which the image was taken, such as illumination, angle, contrast, and scale, which are also present in other biometric recognition systems, e.g. for face. Omara et al BIB005 proposed extracting hierarchical deep features from ear images, fusing the features using discriminant correlation analysis (DCA) Haghighat et al BIB001 to reduce their dimensions, and due to the lack of ear images per person, creating pairwise samples and using pairwise SVM  to perform the matching (since regular SVM would not perform well due to the small size of the datasets). Hansley et al [193] used a fusion of CNNs and handcrafted features for ear recognition which outperformed other state-of-the-art CNN-based works, reaching to the conclusion that handcrafted features can complement deep learning methods.