SpeechEQ: Speech Emotion Recognition based on Multi-scale Unified Datasets and Multitask Learning
Speech emotion recognition (SER) has many challenges, but one of the main
challenges is that each framework does not have a unified standard. In this
paper, we propose SpeechEQ, a framework for unifying SER tasks based on a
multi-scale unified metric. This metric can be trained by Multitask Learning
(MTL), which includes two emotion recognition tasks of Emotion States Category
(EIS) and Emotion Intensity Scale (EIS), and two auxiliary tasks of phoneme
recognition and gender recognition. For this framework, we build a Mandarin SER
dataset - SpeechEQ Dataset (SEQD). We conducted experiments on the public CASIA
and ESD datasets in Mandarin, which exhibit that our method outperforms
baseline methods by a relatively large margin, yielding 8.0\% and 6.5\%
improvement in accuracy respectively. Additional experiments on IEMOCAP with
four emotion categories (i.e., angry, happy, sad, and neutral) also show the
proposed method achieves a state-of-the-art of both weighted accuracy (WA) of
78.16% and unweighted accuracy (UA) of 77.47%.