Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> This paper presents an extension of our previous work which proposes a new speaker representation for speaker verification. In this modeling, a new low-dimensional speaker- and channel-dependent space is defined using a simple factor analysis. This space is named the total variability space because it models both speaker and channel variabilities. Two speaker verification systems are proposed which use this new representation. The first system is a support vector machine-based system that uses the cosine kernel to estimate the similarity between the input data. The second system directly uses the cosine similarity as the final decision score. We tested three channel compensation techniques in the total variability space, which are within-class covariance normalization (WCCN), linear discriminate analysis (LDA), and nuisance attribute projection (NAP). We found that the best results are obtained when LDA is followed by WCCN. We achieved an equal error rate (EER) of 1.12% and MinDCF of 0.0094 using the cosine distance scoring on the male English trials of the core condition of the NIST 2008 Speaker Recognition Evaluation dataset. We also obtained 4% absolute EER improvement for both-gender trials on the 10 s-10 s condition compared to the classical joint factor analysis scoring. <s> BIB001 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> We propose a novel framework for speaker recognition in which extraction of sufficient statistics for the state-of-the-art i-vector model is driven by a deep neural network (DNN) trained for automatic speech recognition (ASR). Specifically, the DNN replaces the standard Gaussian mixture model (GMM) to produce frame alignments. The use of an ASR-DNN system in the speaker recognition pipeline is attractive as it integrates the information from speech content directly into the statistics, allowing the standard backends to remain unchanged. Improvement from the proposed framework compared to a state-of-the-art system are of 30% relative at the equal error rate when evaluated on the telephone conditions from the 2012 NIST speaker recognition evaluation (SRE). The proposed framework is a successful way to efficiently leverage transcribed data for speaker recognition, thus opening up a wide spectrum of research directions. <s> BIB002 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> In this paper we investigate the use of deep neural networks (DNNs) for a small footprint text-dependent speaker verification task. At development stage, a DNN is trained to classify speakers at the frame-level. During speaker enrollment, the trained DNN is used to extract speaker specific features from the last hidden layer. The average of these speaker features, or d-vector, is taken as the speaker model. At evaluation stage, a d-vector is extracted for each utterance and compared to the enrolled speaker model to make a verification decision. Experimental results show the DNN based speaker verification system achieves good performance compared to a popular i-vector system on a small footprint text-dependent speaker verification task. In addition, the DNN based system is more robust to additive noise and outperforms the i-vector system at low False Rejection operating points. Finally the combined system outperforms the i-vector system by 14% and 25% relative in equal error rate (EER) for clean and noisy conditions respectively. <s> BIB003 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> In this paper we present a data-driven, integrated approach to speaker verification, which maps a test utterance and a few reference utterances directly to a single score for verification and jointly optimizes the system's components using the same evaluation protocol and metric as at test time. Such an approach will result in simple and efficient systems, requiring little domain-specific knowledge and making few model assumptions. We implement the idea by formulating the problem as a single neural network architecture, including the estimation of a speaker model on only a few utterances, and evaluate it on our internal"Ok Google"benchmark for text-dependent speaker verification. The proposed approach appears to be very effective for big data applications like ours that require highly accurate, easy-to-maintain systems with a small footprint. <s> BIB004 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> In this paper, we use data augmentation to improve performance of deep neural network (DNN) embeddings for speaker recognition. The DNN, which is trained to discriminate between speakers, maps variable-length utterances to fixed-dimensional embeddings that we call x-vectors. Prior studies have found that embeddings leverage large-scale training datasets better than i-vectors. However, it can be challenging to collect substantial quantities of labeled data for training. We use data augmentation, consisting of added noise and reverberation, as an inexpensive method to multiply the amount of training data and improve robustness. The x-vectors are compared with i-vector baselines on Speakers in the Wild and NIST SRE 2016 Cantonese. We find that while augmentation is beneficial in the PLDA classifier, it is not helpful in the i-vector extractor. However, the x-vector DNN effectively exploits data augmentation, due to its supervised training. As a result, the x-vectors achieve superior performance on the evaluation datasets. <s> BIB005 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> In this paper, we propose a new loss function called generalized end-to-end (GE2E) loss, which makes the training of speaker verification models more efficient than our previous tuple-based end-to-end (TE2E) loss function. Unlike TE2E, the GE2E loss function updates the network in a way that emphasizes examples that are difficult to verify at each step of the training process. Additionally, the GE2E loss does not require an initial stage of example selection. With these properties, our model with the new loss function decreases speaker verification EER by more than 10%, while reducing the training time by 60% at the same time. We also introduce the MultiReader technique, which allows us to do domain adaptation - training a more accurate model that supports multiple keywords (i.e. "OK Google" and "Hey Google") as well as multiple dialects. <s> BIB006 </s> Biometric Recognition Using Deep Learning: A Survey <s> Deep Learning Works on Voice Recognition <s> Learning good representations is of crucial importance in deep learning. Mutual Information (MI) or similar measures of statistical dependence are promising tools for learning these representations in an unsupervised way. Even though the mutual information between two random variables is hard to measure directly in high dimensional spaces, some recent studies have shown that an implicit optimization of MI can be achieved with an encoder-discriminator architecture similar to that of Generative Adversarial Networks (GANs). In this work, we learn representations that capture speaker identities by maximizing the mutual information between the encoded representations of chunks of speech randomly sampled from the same sentence. The proposed encoder relies on the SincNet architecture and transforms raw speech waveform into a compact feature vector. The discriminator is fed by either positive samples (of the joint distribution of encoded chunks) or negative samples (from the product of the marginals) and is trained to separate them. We report experiments showing that this approach effectively learns useful speaker representations, leading to promising results on speaker identification and verification tasks. Our experiments consider both unsupervised and semi-supervised settings and compare the performance achieved with different objective functions. <s> BIB007
Before the era of deep learning, most state-of-theart speaker recognition systems are built with the ivectors approach BIB001 , which uses factor analysis to define a low-dimensional space that models both speaker and channel variabilities. In recent years, it has become more and more popular to explore deep learning approaches for speaker recognition. One of the first approach among these efforts is to incorporate DNN-based acoustic models into the i-vector framework BIB002 . This method uses an DNN acoustic model trained for Automatic Speech Recognition (ASR) to gather speaker statistics for i-vector model training. It has been shown that this improvement leads to a 30% relative reduction in equal error rate. Around the same time, d-vector was proposed in BIB003 to tackle text-dependent speaker recognition using neural network. In this approach, a DNN is trained to classify speakers at the frame-level. During enrollment and testing, the trained DNN is used to extract speaker specific features from the last hidden layer. dvectors are then computed by averaging these features and used as speaker embeddings for recognition. This method shows 14% and 25% relative improvement over an i-vector system under clean and noisy conditions, respectively. In BIB005 , a time-delay neural network is trained to extract segment level x-vectors for text-independent speech recognition. This network takes in features of speech segments and passes them through a few non-linear layers followed by a pooling layer to classify speakers at segment-level. X-vectors are then extracted from the pooling layer for enrollment and testing. It is shown that an x-vector system can achieve a better speaker recognition performance compared to the traditional ivector approach, with the help of data augmentation. End-to-end approaches based on neural networks are also explored in various papers. In BIB004 and  , neural networks are designed to take in pairs of speech segments, and are trained to classify match/mismatch targets. A specially designed triplet loss function is proposed in  to substitute a binary classification loss function. Generalized end-to-end (GE2E) loss, which is similar to triplet loss, is proposed in BIB006 for textdependent speaker recognition on an in-house dataset. In  , a complementary optimizing goal called intra-class loss is proposed to improve deep speaker embeddings learned with triplet loss. It is shown in the paper that models trained using intra-class loss can yield a significant relative reduction of 30% in equal error rate (EER) compared to the original triplet loss. The effectiveness is evaluated on both VoxCeleb and VoxForge datasets. In BIB007 , the authors proposed a method for learning speaker embeddings from raw waveform by maximizing the mutual information. This approach uses an encoderdiscriminator architecture similar to that of Generative Adversarial Networks (GANs) to optimize mutual information implicitly. The authors show that this approach effectively learns useful speaker representations, leading to a superior performance on the VoxCeleb corpus when compared with i-vector baseline and CNN-based triples loss systems. In  , the authors combine a deep convolutional feature extractor, self-attentive pooling and large-margin loss functions into their end-to-end deep speaker recognizers. The individual and ensemble models from this approach achieved state-of-the-art performance on Vox-Celeb with a relative improvement of 70% and 82%, respectively, over the best reported results. The authors also proposed to use a neural network to subsitute PLDA classifier, which enables them to get the state-of-the-art results on NIST-SRE 2016 dataset.