In this work, we investigate transfer learning between three
paralinguistic tasks: emotion, speaker, and gender
recognition, with a focus on emotion recognition as the target domain.
In addition, we investigate the efficacy of transfer learning applied between two
emotion datasets: IEMOCAP [1]} and MSP-IMPROV [2]}.
Finally, we study the effect of transfer learning between datasets when the target task
has limited amount of data available.
In all cases, we investigate three methods: (1) deep neural network (DNN); (2) DNN
with PT/FT; and (3) progressive networks. Our results demonstrate significant improvements over the conventional PT/FT methods when using ProgNets for transferring knowledge from speaker recognition to emotion recognition tasks. Furthermore, our results suggest that ProgNets show promise as a method for transferring knowledge from gender detection to emotion recognition tasks, as well as transferring knowledge across datasets, with results significantly better than a DNN without transfer learning.
