In this paper the task of emotion recognition from speech is considered. Proposed approach uses deep recurrent neural network trained on a sequence of acoustic features calculated over small speech intervals. At the same time special probabilistic-nature CTC loss function allows to consider long utterances containing both emotional and neutral parts. The effectiveness of such an approach is shown in two ways. Firstly, the comparison with recent advances in this field is carried out. Secondly, human performance on the same task is measured. Both criteria show the high quality of the proposed method.
One of the first deep learning end-to-end approaches was presented by in 2014 in their work @cite_38 . Their idea is to split each utterance into frames and calculate low-level features at the first step. Then authors used densely connected neural network with three hidden layers to transform this sequence of features to the sequence of probability distributions over the target emotion labels. Then these probabilities are aggregated into utterance-level features using simple statistics like maximum, minimum, average, percentiles, etc. After that the Extreme Learning Machine (ELM) @cite_34 is trained to classify utterances by emotional state.
@cite_38 @cite_34
2295001676 2111072639
Abstract Speech emotion recognition is a challenging problem partly be-cause it is unclear what features are effective for the task. Inthis paper we propose to utilize deep neural networks (DNNs)to extract high level features from raw data and show that theyare effective for speech emotion recognition. We ﬁrst producean emotion state probability distribution for each speech seg-ment using DNNs. We then construct utterance-level featuresfrom segment-level probability distributions. These utterance-level features are then fed into an extreme learning machine(ELM),aspecialsimpleandefﬁcientsingle-hidden-layer neuralnetwork, to identify utterance-level emotions. The experimen-tal results demonstrate that the proposed approach effectivelylearns emotional information from low-level features and leadsto 20 relative accuracy improvement compared to the state-of-the-art approaches. IndexTerms : Emotion recognition, Deep neural networks, Ex-treme learning machine 1. Introduction Despite the great progress made in artiﬁcial intelligence, weare still far from being able to naturally interact with machines,partly because machines do not understand our emotion states.Recently, speech emotion recognition, which aims to recognizeemotion states from speech signals, has been drawing increas-ing attention. Speech emotion recognition is a very challengingtask of which extracting effective emotional features is an openquestion [1, 2].Adeep neural network (DNN) is afeed-forward neural net-work that has more than one hidden layers between its inputsand outputs. It is capable of learning high-level representationfrom the raw features and effectively classifying data [3, 4].With sufﬁcient training data and appropriate training strategies,DNNs perform very well in many machine learning tasks (e.g.,speech recognition [5]).Feature analysis in emotion recognition is much less stud-ied than that in speech recognition. Most previous studies em-pirically chose features for emotion classiﬁcation. In this study,aDNNtakesasinputtheconventional acousticfeatureswithinaspeechsegmentandproducessegment-levelemotionstateprob-ability distributions, from which utterance-level features areconstructed and used to determine the utterance-level emotionstate. Since the segment-level outputs already provide consid-erable emotional information and the utterance-level classiﬁca- Abstract It is clear that the learning speed of feedforward neural networks is in general far slower than required and it has been a major bottleneck in their applications for past decades. Two key reasons behind may be: (1) the slow gradient-based learning algorithms are extensively used to train neural networks, and (2) all the parameters of the networks are tuned iteratively by using such learning algorithms. Unlike these conventional implementations, this paper proposes a new learning algorithm called e xtreme l earning m achine (ELM) for s ingle-hidden l ayer f eedforward neural n etworks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs. In theory, this algorithm tends to provide good generalization performance at extremely fast learning speed. The experimental results based on a few artificial and real benchmark function approximation and classification problems including very large complex applications show that the new algorithm can produce good generalization performance in most cases and can learn thousands of times faster than conventional popular learning algorithms for feedforward neural networks. 1