Employing voice-based emotion recognition function in artificial intelligence (AI) product will improve the user experience. Most of researches that have been done only focus on the speech collected under controlled conditions. The scenarios evaluated in these research were well controlled. The conventional approach may fail when background noise or nonspeech filler exist. In this paper, we propose an ensemble framework combining several aspects of features from audio. The framework incorporates gender and speaker information relying on multi-task learning. Therefore it is able to dig and capture emotional information as much as possible. This framework is evaluated on multimodal emotion challenge (MEC) 2017 corpus which is close to real world. The proposed framework outperformed the best baseline system by 29.5 (relative improvement).
Abstract Most paralinguistic analysis tasks are lacking agreed-uponevaluation procedures and comparability, in contrast to more‘traditional’ disciplines in speech analysis. The INTERSPEECH2010 Paralinguistic Challenge shall help overcome the usuallylow compatibility of results, by addressing three selected sub-challenges. In the Age Sub-Challenge, the age of speakers hasto be determined in four groups. In the Gender Sub-Challenge,a three-class classiﬁcation task has to be solved and ﬁnally, theAffect Sub-Challenge asks for speakers’ interest in ordinal rep-resentation. This paper introduces the conditions, the Challengecorpora “aGender” and “TUM AVIC” and standard feature setsthat may be used. Further, baseline results are given.Index Terms: Paralinguistic Challenge, Age, Gender, Affect 1. Introduction Most paralinguistic analysis tasks resemble each other not onlyby means of processing and ever-present data sparseness, but bylacking agreed-upon evaluation procedures and comparability,in contrast to more traditional disciplines in speech analysis. Atthe same time, this is a rapidly emerging ﬁeld of research, dueto the constantly growing interest on applications in the ﬁeldsof Human-Machine Communication, Human-Robot Communi-cation, and Multimedia Retrieval. In these respects, the INTER-SPEECH 2010 Paralinguistic Challenge shall help bridging thegap between excellent research on paralinguistic information inspoken language and low compatibility of results, by address-ing three selected tasks. The “aGender” and the “TUM AVIC”corpora are provided by the organizers. The ﬁrst consists of 46hours of telephone speech, stemming from 954 speakers, andserves to evaluate features and algorithms for the detection ofspeaker age and gender. The second features 2 hours of humanconversational speech recording (21 subjects), annotated in 5differentlevelsofinterest. Thecorpusfurtherfeaturesauniquelydetailed transcription of spoken content with word boundaries byforced alignment, non-linguistic vocalizations, single annotatortracks, and the sequence of (sub-)speaker-turns. Both are given A utility stool having vertically spaced end frames connected by a generally U-shaped body member that is detachably connected to the frames in either of two vertically reversed positions, and at various elevations in either of such positions, together with means for supporting a variety of attachments. Recognizing human emotions attitudes from speech cues has gained increased attention recently. Most previous work has focused primarily on suprasegmental prosodic features calculated at the utterance level for this purpose. Notably, not much attention is paid to details at the segmental phoneme level in the modeling. Based on the hypothesis that different emotions have varying effects on the properties of the different speech sounds, this paper investigates the usefulness of phoneme-level modeling for the classification of emotional states from speech. Hidden Markov models (HMM) based on short-term spectral features are used for this purpose using data obtained from a recording of an actress’ expressing 4 different emotional states anger, happiness, neutral, and sadness. We designed and compared two sets of HMM classifiers: a generic set of ”emotional speech” HMMs (one for each emotion) a set of broad phoneticclass based HMMs for each emotion type considered. Five broad phonetic classes were used to explore the effect of emotional coloring on different phoneme classes, and it was found that (spectral properties of) vowel sounds were the best indicator to emotions in terms of the classification performance. The experiments also showed that the best performance can be obtained by using phoneme-class classifiers over generic “emotional” HMM classifier and classifiers based on global prosodic features. To see complementary effect of the prosodic and spectral features, two classifiers were combined at the decision level. The improvement was 0.55 in absolute compared with the result from phoneme-class based HMM classifier. In this contribution we introduce speech emotion recognition by use of continuous hidden Markov models. Two methods are propagated and compared throughout the paper. Within the first method a global statistics framework of an utterance is classified by Gaussian mixture models using derived features of the raw pitch and energy contour of the speech signal. A second method introduces increased temporal complexity applying continuous hidden Markov models considering several states using low-level instantaneous features instead of global statistics. The paper addresses the design of working recognition engines and results achieved with respect to the alluded alternatives. A speech corpus consisting of acted and spontaneous emotion samples in German and English language is described in detail. Both engines have been tested and trained using this equivalent speech corpus. Results in recognition of seven discrete emotions exceeded 86 recognition rate. As a basis of comparison the similar judgment of human deciders classifying the same corpus at 79.8 recognition rate was analyzed. In this paper we explore one of the key aspects in building an emotion recognition system: generating suitable feature representations. We generate feature representations from both acoustic and lexical levels. At the acoustic level, we first extract low-level features such as intensity, F0, jitter, shimmer and spectral contours etc. We then generate different acoustic feature representations based on these low-level features, including statistics over these features, a new representation derived from a set of low-level acoustic codewords, and a new representation from Gaussian Supervectors. At the lexical level, we propose a new feature representation named emotion vector (eVector). We also use the traditional Bag-of-Words (BoW) feature. We apply these feature representations for emotion recognition and compare their performance on the USC-IEMOCAP database. We also combine these different feature representations via early fusion and late fusion. Our experimental results show that late fusion of both acoustic and lexical features achieves four-class emotion recognition accuracy of 69.2 .