The standard approach to semi-supervised training uses an initial model to transcribe the unlabeled data and then treats the hypothesized transcripts as truth.
We start with the supervised model as our transcription model.
During SST we use both the supervised data and the untranscribed data, making no distinction between the two during the training process.
For hybrid models keeping only a subset of the hypothesized transcripts can be important.
The common approach is to simply filter by confidence, but more sophisticated methods are sometimes used [1]}, [2]}.
For autoregressive models, we have found selection to be less of an issue.
The models are data hungry and tend to perform better with more data, even if it is lower quality.
