Self-supervised learning is a clever way to learn general information from data without requiring any labels. Recently many successful methods have emerged for self-supervised representation learning from speech. The general idea is to implicitly learn the global structure and local characteristics that are inherently present in speech. Depending on the task, both local information, such as the pronunciation of a specific phoneme, and more global information, such as speaker traits and recording properties, can be useful. By pre-training a network with a well-chosen objective function, these relevant attributes about the input speech can be captured and summarised in rich feature vectors. This improves several downstream tasks like speech recognition and typically reduces the amount of required data, since the principal characteristics are already extracted and more easily accessible. Moreover, the structure of a speech waveform is to some extent general and language-independent, which explains the improvements with these features in low-resource languages [1]}, [2]}.
