Multitask vocal burst modeling with ResNets and pre-trained paralinguistic Conformers
This technical report presents the modeling approaches used in our submission
to the ICML Expressive Vocalizations Workshop & Competition multitask track
(ExVo-MultiTask). We first applied image classification models of various sizes
on mel-spectrogram representations of the vocal bursts, as is standard in sound
event detection literature. Results from these models show an increase of
21.24% over the baseline system with respect to the harmonic mean of the task
metrics, and comprise our team's main submission to the MultiTask track. We
then sought to characterize the headroom in the MultiTask track by applying a
large pre-trained Conformer model that previously achieved state-of-the-art
results on paralinguistic tasks like speech emotion recognition and mask
detection. We additionally investigated the relationship between the sub-tasks
of emotional expression, country of origin, and age prediction, and discovered
that the best performing models are trained as single-task models, questioning
whether the problem truly benefits from a multitask setting.