We further demonstrate that these continuous and discrete labels need not necessarily be manually annotated within the same corpus to be improve recognition performance. Of the prevailing annotated datasets for emotion recognition, very few are annotated for both continuous and discrete attributes such as MSPPodcast [1]}, and IEMOCAP[2]} while others like MELD[3]} are annotated for discrete attributes only. We demonstrate that even if the model is trained on continuous emotion labels from MSPPodcast and discrete labels from IEMOCAP, the proposed Hierarchical Multi-Task approach improves performance and generalizability. This implies that emotion recognition datasets can be trained jointly on multiple corpora with different labels. In summary, this paper makes the following contributions:
