Self-supervised learning (SSL) enables the learning of meaningful and disentangled features from unlabeled speech data that can be used across different tasks. These representations are becoming essential and are now part of several state-of-the-art speech applications [1]}, [2]}. Widely-used representations include wav2vec 2.0 [1]} and HuBERT [2]}. The latter is an SSL model that extracts features directly from the raw speech signal. Via an offline k-means clustering step, the model learns combined acoustic and language representations from masked inputs using a BERT-like reconstruction loss.
