Building a good speech recogniser typically requires a large amount of annotated data from a specific language. Obtaining high-quality labelled data is a costly and time-intensive process, and for many languages this remains a big issue. However, even in a highly resourced language like English, recent work has shown impressive results in Automatic Speech Recognition (ASR) by pre-training on unlabelled data and transferring that knowledge to regular speech recognition models [1]}, [2]}, [3]}
or even completely unsupervised speech recognition [4]}.
This paradigm shift towards unlabelled data is of great significance as untranscribed recordings of speech are much easier to acquire.
