We employ the RoBERTa model [1]} to extract context independent utterance level feature vectors.
We first fine-tune the RoBERTa Large model for emotion label prediction from the transcript of the utterances. RoBERTa Large follows the original BERT Large [2]} architecture having 24 layers, 16 self-attention heads in each block and a hidden dimension of 1024, resulting in a total of 355M parameters. Let an utterance \(x\)  consists of a sequence of BPE tokenized tokens \(x_1, x_2, \dots , x_N\) , with emotion label \(E_x\) . In this setting, the fine-tuning of the pretrained RoBERTa model is realized through a sentence classification task. A special token \([CLS]\)  is appended at the beginning of the utterance to create the input sequence for the model: \([CLS], x_1, x_2, \dots , x_N\) . This sequence is passed through the model, and the activation from the last layer corresponding to the \([CLS]\)  token is then used in a small feedforward network to classify it into its emotion class \(E_x\) .
