Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning
Emotion classification of speech and assessment of the emotion strength are
required in applications such as emotional text-to-speech and voice conversion.
The emotion attribute ranking function based on Support Vector Machine (SVM)
was proposed to predict emotion strength for emotional speech corpus. However,
the trained ranking function doesn't generalize to new domains, which limits
the scope of applications, especially for out-of-domain or unseen speech. In
this paper, we propose a data-driven deep learning model, i.e. StrengthNet, to
improve the generalization of emotion strength assessment for seen and unseen
speech. This is achieved by the fusion of emotional data from various domains.
We follow a multi-task learning network architecture that includes an acoustic
encoder, a strength predictor, and an auxiliary emotion predictor. Experiments
show that the predicted emotion strength of the proposed StrengthNet is highly
correlated with ground truth scores for both seen and unseen speech. We release
the source codes at: https://github.com/ttslr/StrengthNet.