wav2vec 2.0 was shown to obtain good baseline results for this challenge in SSL-MOS [1]}. The speech waveform is input to wav2vec 2.0. For wav2vec 2.0, the public (available on the 'fairseq' GitHub repository [2]}) 95M-parameter 'base' model pretrained on LibriSpeech was used, and an intermediate layer's 768-dimensional latent space with a variable number of frames was extracted. Several convolutional and pooling layers remove the time dimension and reduce it down to 64-dimensions. For both models with and without wav2vec 2.0, the one-hot metadata features described above are concatenated to the pooled wav2vec 2.0 64-dimensional features, and four additional fully connected layers reduce it to a 1-dimensional MOS. This predicted MOS is used with an L1 loss on the subjective MOS.
