A common approach to incorporating untranscibed data in ASR training is through semi-supervised training (SST).
A strong lexicon and language model—provided externally to a hybrid model or implicit in a sequence-to-sequence model—restricts the output to sequences of likely words in the language.
Because of the extra information added by the language model, SST works remarkably well for ASR.
When trying to improve a well-trained supervised model, it can take an order of magnitude more unsupervised data to provide an improvement [1]}, [2]}.
In the case of domain adaptation, SST can produce large gains with just a few hours of data [3]}.
