Identity recognition from ear images is an active field of research within the biometric community. The ability to capture ear images from a distance and in a covert manner makes ear recognition technology an appealing choice for surveillance and security applications as well as related application domains. In contrast to other biometric modalities, where large datasets captured in uncontrolled settings are readily available, datasets of ear images are still limited in size and mostly of laboratory-like quality. As a consequence, ear recognition technology has not benefited yet from advances in deep learning and convolutional neural networks (CNNs) and is still lacking behind other modalities that experienced significant performance gains owing to deep recognition technology. In this paper we address this problem and aim at building a CNNbased ear recognition model. We explore different strategies towards model training with limited amounts of training data and show that by selecting an appropriate model architecture, using aggressive data augmentation and selective learning on existing (pre-trained) models, we are able to learn an effective CNN-based model using a little more than 1300 training images. The result of our work is the first CNN-based approach to ear recognition that is also made publicly available to the research community. With our model we are able to improve on the rank one recognition rate of the previous state-of-the-art by more than 25 on a challenging dataset of ear images captured from the web (a.k.a. in the wild).
The last groups of techniques, so-called hybrid techniques, combine elements from other categories or use multiple representations to increase the ear recognition performance @cite_12 . Techniques from this group offer superior performance compared to competing techniques, but often at the cost of higher computational complexity @cite_15 @cite_0 @cite_13 . As suggested by recent surveys on ear recognition @cite_12 @cite_17 @cite_32 , hybrid techniques together with local descriptor-based methods represent the current state-of-the-art in this area.
@cite_32 @cite_0 @cite_15 @cite_13 @cite_12 @cite_17
2019999542 2158057578 2134533334 2139709289 2401484698 2770960269 2062690810 335504665 2156336287 1874503286 2170415971 2734984521 1652729136 2012596828 2121981798 1982048725 2735835382 2787721211 2043287655 2962984063
If fusion rules cannot adapt to the changes of environment and individual users, multimodal systems may perform worse than unimodal systems when one or more modalities encounter data degeneration. This paper develops a robust face and ear based multimodal biometric system using Sparse Representation (SR), which integrates the face and ear at feature level, and can effectively adjust the fusion rule based on reliability difference between the modalities. We first propose a novel index called Sparse Coding Error Ratio (SCER) to measure the reliability difference between face and ear query samples. Then, SCER is utilized to develop an adaptive feature weighting scheme for dynamically reducing the negative effect of the less reliable modality. In multimodal classification phase, SR-based classification techniques are employed, i.e., Sparse Representation based Classification (SRC) and Robust Sparse Coding (RSC). Finally, we derive a category of SR-based multimodal recognition methods, including Multimodal SRC with feature Weighting (MSRCW) and Multimodal RSC with feature Weighting (MRSCW). Experimental results demonstrate that: (a) MSRCW and MRSCW perform significantly better than the unimodal recognition using either face or ear alone, as well as the known multimodal methods; (b) The effectiveness of adaptive feature weighting is verified. MSRCW and MRSCW are very robust to the image degeneration occurring to one of the modalities. Even when face (ear) query sample suffers from 100 random pixel corruption, they can still get the performance close to the ear (face) unimodal recognition; (c) By integrating the advantages of adaptive feature weighting and sparsity-constrained regression, MRSCW seems excellent in tackling the face and ear based multimodal recognition problem. Ear recognition is a new biometrics technique. Due to its unique physiological structure, position and stability, ear recognition is expected to be a promising authentication technique. In this paper, a hybrid system for classifying ear images is proposed. This system combines independent component analysis (ICA) and RBF network. The original ear image database is decomposed into linear combinations of several basic images. Then the corresponding coefficients of these combinations are fed up into RBF network instead of an original feature vector comprised of pixel values of grayscale images. The local features extraction of ICA and the adaptability of RBF neural network are combined reasonably. The robustness of the system is enhanced. The experiment results show that the recognition rate of ICA RBF method is improved substantially. The use of multiple distant microphones has been widely studied for meeting recognition. The two most widely used approaches are 1) combination at the signal level, via blind beamforming, followed by recognition of a single enhanced audio signal, and 2) independent, logically parallel recognition of the multiple audio sources followed by hypothesis-level combination. In this paper we investigate how these two approaches compare for state-of-the-art recognition systems applied to meeting data from the two most recent NIST Rich Transcription evaluations. Our results show that beamforming is the superior approach, giving more accurate results while being inherently less computationally demanding. We then propose a hybrid approach that leverages both beamforming and signal-level diversity for systemcombination, and showthat this approach gives gains over either of the old methods. The amount of music information available on the Web is rapidly increasing. There is a pressing need for music information extraction. To extract useful information from natural language text, we must recognize music named entities first. This paper introduces a hybrid method to identify the Chinese named entities in music domain. Recently, machine learning approaches are frequently used to solve Name Entity Recognition (NER). So our method uses a Hidden Markov Model (HMM) as the underlying method. Since HMM has innate weaknesses, we incorporate it with rule-based method for pre-processing and post-processing. The combination of machine learning method and rule-based method results in a high precision recognition. And we improve both training and recognizing process of HMM for Music Named Entity Recognition (MNER). In this paper, a novel and convenient Musical Name Entity (MNE) tagging method to generate training data is proposed, which makes HMM method practically usable. In addition, we present an effective method of unknown words tagging in recognition. The experimental results show that our framework brings significant improvements for solving MNER. A novel multi-focus image fusion approach using coupled dictionary training is proposed. It exploits the facts that (i) the patches in example data can be sparsely represented by a couple of over-complete dictionaries related to the focused and blurred categories of images and (ii) merging such representations is better than just selecting the sparsest one in the estimate of the original image. Inspired by these observations, we enforce the similarity of sparse representations between the focused and blurred image patches by jointly training the coupled dictionary, and then fuse these representations to generate an all-in-focus image by a fusion rule. The key characteristics of our approach are bridging the gap between coupled dictionaries, combining plain averaging and "choose-max" as an appropriate fusion rule, and forming a more accurate representation, compared to existing approaches which simply admit sparse representation over one dictionary. Extensive experimental comparisons with state-of-the-art multi-focus image fusion algorithms validate the effectiveness of the proposed approach. Protection of data integrity and person identity has been an active research area for many years. Among the techniques investigated, developing multi-modal recognition systems using audio and face signals for people authentication holds a promising future due to its ease of use. A challenge in developing such a multi-modal recognition system is to improve its reliability for a practical application. In this paper, an efficient audio-visual bimodal recognition system which uses Deep Convolution Neural Networks (CNNs) as a primary model architecture. First, two separate Deep CNN models are trained with the help of audio and facial features, respectively. The outputs of these CNN models are then combined fused to predict the identity of the subject. Implementation details with regard to data fusion are discussed in a great length in the paper. Through experimental verification, the proposed bimodal fusion approach is superior in accuracy performance when compared with any single modal recognition systems and with published results using the same data-set. Recognizing people by their ear has recently received significant attention in the literature. Several reasons account for this trend: first, ear recognition does not suffer from some problems associated with other non-contact biometrics, such as face recognition; second, it is the most promising candidate for combination with the face in the context of multi-pose face recognition; and third, the ear can be used for human recognition in surveillance videos where the face may be occluded completely or in part. Further, the ear appears to degrade little with age. Even though current ear detection and recognition systems have reached a certain level of maturity, their success is limited to controlled indoor conditions. In addition to variation in illumination, other open research problems include hair occlusion, earprint forensics, ear symmetry, ear classification, and ear individuality. This article provides a detailed survey of research conducted in ear detection and recognition. It provides an up-to-date review of the existing literature revealing the current state-of-art for not only those who are working in this area but also for those who might exploit this new approach. Furthermore, it offers insights into some unsolved ear recognition problems as well as ear databases available for researchers. In this paper we present a novel solution to the Overdetermined Blind Speech Separation (OBSS) problem for improving speech recognition accuracy of N simultaneous speakers in real room environments using M, (M>N) microphones. The proposed OBSS system uses basic NxN Blind Speech Separation networks that process in parallel all different combinations of the available mixture signals in the frequency domain, resulting to lower computational complexity and faster convergence. Extensive experiments using an array of two to ten microphones and two simultaneous speakers in a simulated real room, showed that when the number of the microphones increases beyond two, the separation performance is improved and the phoneme recognition accuracy of an HMM based decoder increases drastically (more than 6 ). Therefore, the introduction of more microphones than speakers is justified in order to improve speech recognition accuracy in multi simultaneous speaker environments. We develop a novel technique for class-based matching of object parts across large changes in viewing conditions. Given a set of images of objects from a given class under different viewing conditions, the algorithm identifies corresponding regions depicting the same object part in different images. The technique is based on using the equivalence of corresponding features in different viewing conditions. This equivalence-based matching scheme is not restricted to planar components or affine transformations. As a result, it identifies corresponding parts more accurately and under more general conditions than previous methods. The scheme is general and works for a variety of natural object classes. We demonstrate that using the proposed methods, a dense set of accurate correspondences can be obtained. Experimental comparisons to several known techniques are presented. An application to the problem of invariant object recognition is shown, and additional applications to wide-baseline stereo are discussed. Existing techniques for 3D action recognition are sensitive to viewpoint variations because they extract features from depth images which change significantly with viewpoint. In contrast, we directly process the pointclouds and propose a new technique for action recognition which is more robust to noise, action speed and viewpoint variations. Our technique consists of a novel descriptor and keypoint detection algorithm. The proposed descriptor is extracted at a point by encoding the Histogram of Oriented Principal Components (HOPC) within an adaptive spatio-temporal support volume around that point. Based on this descriptor, we present a novel method to detect Spatio-Temporal Key-Points (STKPs) in 3D pointcloud sequences. Experimental results show that the proposed descriptor and STKP detector outperform state-of-the-art algorithms on three benchmark human activity datasets. We also introduce a new multiview public dataset and show the robustness of our proposed method to viewpoint variations. We present results of the largest experimental investigation of ear biometrics to date. Approaches considered include a PCA ("eigen-ear") approach with 2D intensity images, achieving 63.8 rank-one recognition; a PCA approach with range images, achieving 55.3 Hausdorff matching of edge images from range images, achieving 67.5 and ICP matching of the 3D data, achieving 98.7 . ICP based matching not only achieves the best performance, but also shows good scalability with size of dataset. The data set used represents over 300 persons, each with images acquired on at least two different dates. In addition, the ICP-based approach is further applied on an expanded data set of 404 subjects, and achieves 97.5 rank one recognition rate. In order to test the robustness and variability of ear biometrics, ear symmetry is also investigated. In our experiments around 90 of peoples right ear and left ear are symmetric. Audio–visual recognition (AVR) has been considered as a solution for speech recognition tasks when the audio is corrupted, as well as a visual recognition method used for speaker verification in multi-speaker scenarios. The approach of AVR systems is to leverage the extracted information from one modality to improve the recognition ability of the other modality by complementing the missing information. The essential problem is to find the correspondence between the audio and visual streams, which is the goal of this paper. We propose the use of a coupled 3D convolutional neural network (3D CNN) architecture that can map both modalities into a representation space to evaluate the correspondence of audio–visual streams using the learned multimodal features. The proposed architecture will incorporate both spatial and temporal information jointly to effectively find the correlation between temporal information for different modalities. By using a relatively small network architecture and much smaller data set for training, our proposed method surpasses the performance of the existing similar methods for audio–visual matching, which use 3D CNNs for feature representation. We also demonstrate that an effective pair selection method can significantly increase the performance. The proposed method achieves relative improvements over 20 on the equal error rate and over 7 on the average precision in comparison to the state-of-the-art method. The paper addresses the problem of speaker (or voice) de-identification by presenting a novel approach for concealing the identity of speakers in their speech. The proposed technique first recognizes the input speech with a diphone recognition system and then transforms the obtained phonetic transcription into the speech of another speaker with a speech synthesis system. Due to the fact that a Diphone RecOgnition step and a sPeech SYnthesis step are used during the de-identification, we refer to the developed technique as DROPSY. With this approach the acoustical models of the recognition and synthesis modules are completely independent from each other, which ensures the highest level of input speaker de-identification. The proposed DROPSY-based de-identification approach is language dependent, text independent and capable of running in real-time due to the relatively simple computing methods used. When designing speaker de-identification technology two requirements are typically imposed on the de-identification techniques: i) it should not be possible to establish the identity of the speakers based on the de-identified speech, and ii) the processed speech should still sound natural and be intelligible. This paper, therefore, implements the proposed DROPSY-based approach with two different speech synthesis techniques (i.e, with the HMM-based and the diphone TD-PSOLA-based technique). The obtained de-identified speech is evaluated for intelligibility and evaluated in speaker verification experiments with a state-of-the-art (i-vector PLDA) speaker recognition system. The comparison of both speech synthesis modules integrated in the proposed method reveals that both can efficiently de-identify the input speakers while still producing intelligible speech. During the last years a wide range of algorithms and devices have been made available to easily acquire range images. The increasing abundance of depth data boosts the need for reliable and unsupervised analysis techniques, spanning from part registration to automated segmentation. In this context, we focus on the recognition of known objects in cluttered and incomplete 3D scans. Locating and fitting a model to a scene are very important tasks in many scenarios such as industrial inspection, scene understanding, medical imaging and even gaming. For this reason, these problems have been addressed extensively in the literature. Several of the proposed methods adopt local descriptor-based approaches, while a number of hurdles still hinder the use of global techniques. In this paper we offer a different perspective on the topic: We adopt an evolutionary selection algorithm that seeks global agreement among surface points, while operating at a local level. The approach effectively extends the scope of local descriptors by actively selecting correspondences that satisfy global consistency constraints, allowing us to attack a more challenging scenario where model and scene have different, unknown scales. This leads to a novel and very effective pipeline for 3D object recognition, which is validated with an extensive set of experiments and comparisons with recent techniques at the state of the art. A trend in automatic speech recognition systems is the use of continuous mixture-density hidden Markov models (HMMs). Despite the good recognition performance that these systems achieve on average in large vocabulary applications, there is a large variability in performance across speakers. Performance degrades dramatically when the user is radically different from the training population. A popular technique that can improve the performance and robustness of a speech recognition system is adapting speech models to the speaker, and more generally to the channel and the task. In continuous mixture-density HMMs the number of component densities is typically very large, and it may not be feasible to acquire a sufficient amount of adaptation data for robust maximum-likelihood estimates. To solve this problem, the authors propose a constrained estimation technique for Gaussian mixture densities. The algorithm is evaluated on the large-vocabulary Wall Street Journal corpus for both native and nonnative speakers of American English. For nonnative speakers, the recognition error rate is approximately halved with only a small amount of adaptation data, and it approaches the speaker-independent accuracy achieved for native speakers. For native speakers, the recognition performance after adaptation improves to the accuracy of speaker-dependent systems that use six times as much training data. > We present a novel approach to address the representation issue and the matching issue in face recognition (verification). Firstly, our approach encodes the micro-structures of the face by a new learning-based encoding method. Unlike many previous manually designed encoding methods (e.g., LBP or SIFT), we use unsupervised learning techniques to learn an encoder from the training examples, which can automatically achieve very good tradeoff between discriminative power and invariance. Then we apply PCA to get a compact face descriptor. We find that a simple normalization mechanism after PCA can further improve the discriminative ability of the descriptor. The resulting face representation, learning-based (LE) descriptor, is compact, highly discriminative, and easy-to-extract. To handle the large pose variation in real-life scenarios, we propose a pose-adaptive matching method that uses pose-specific classifiers to deal with different pose combinations (e.g., frontal v.s. frontal, frontal v.s. left) of the matching face pair. Our approach is comparable with the state-of-the-art methods on the Labeled Face in Wild (LFW) benchmark (we achieved 84.45 recognition rate), while maintaining excellent compactness, simplicity, and generalization ability across different datasets. Abstract As one of the most challenging and attractive problems in the pattern recognition and machine intelligence field, imbalanced classification has received a large amount of research attention for many years. In binary classification tasks, one class usually tends to be underrepresented when it consists of far fewer patterns than the other class, which results in undesirable classification results, especially for the minority class. Several techniques, including resampling, boosting and cost-sensitive methods have been proposed to alleviate this problem. Recently, some ensemble methods that focus on combining individual techniques to obtain better performance have been observed to present better classification performance on the minority class. In this paper, we propose a novel ensemble framework called Adaptive Ensemble Undersampling-Boost for imbalanced learning. Our proposal combines the Ensemble of Undersampling (EUS) technique, Real Adaboost, cost-sensitive weight modification, and adaptive boundary decision strategy to build a hybrid algorithm. The superiority of our method over other state-of-the-art ensemble methods is demonstrated by experiments on 18 real world data sets with various data distributions and different imbalance ratios. Given the experimental results and further analysis, our proposal is proven to be a promising alternative that can be applied to various imbalanced classification domains. State-of-the-art Optical Music Recognition system often fails to process dense and damaged music scores, where many symbols can present complex segmentation problems. We propose to resolve these segmentation problems by using a CNN-based detector trained with few manually annotated data. A data augmentation bootstrapping method is used to accurately train a deep learning model to do the localization and classification of an accidental symbol associated with a note head, or the note head if there is no accidental. Using 5-fold cross-validation, we obtain an average of 98.5 localization with an IoU score over 0.5 and a classification accuracy of 99.2 . In this work we propose a local approach of 2D ear authentication. A multi-matcher system is proposed where each matcher is trained using features extracted from a single sub-window of the whole 2D image. The features are extracted by the convolution of each sub-window with a bank of Gabor Filters, then their dimensionality is reduced by Laplacian EigenMaps. The best matchers, corresponding to the most discriminative sub-windows, are selected by running the Sequential Forward Floating Selection (SFFS). Our experiments, carried out on a database of 114 people, show that combining only few ( ten) sub-windows in the fusion step it is possible to achieve a very low Equal Error Rate. Abstract Motivated by the success of powerful while expensive techniques to recognize words in a holistic way (, 2013; , 2014; , 2016) object proposals techniques emerge as an alternative to the traditional text detectors. In this paper we introduce a novel object proposals method that is specifically designed for text. We rely on a similarity based region grouping algorithm that generates a hierarchy of word hypotheses. Over the nodes of this hierarchy it is possible to apply a holistic word recognition method in an efficient way. Our experiments demonstrate that the presented method is superior in its ability of producing good quality word proposals when compared with class-independent algorithms. We show impressive recall rates with a few thousand proposals in different standard benchmarks, including focused or incidental text datasets, and multi-language scenarios. Moreover, the combination of our object proposals with existing whole-word recognizers (, 2014; , 2016) shows competitive performance in end-to-end word spotting, and, in some benchmarks, outperforms previously published results. Concretely, in the challenging ICDAR2015 Incidental Text dataset, we overcome in more than 10 F -score the best-performing method in the last ICDAR Robust Reading Competition (Karatzas, 2015). Source code of the complete end-to-end system is available at https: github.com lluisgomez TextProposals .