However, a strong pseudotranscription seed model is very important for semisupervised training, especially in the case of seq2seq models [1]}.
While work has been done on using pseudotranscripts in seq2seq models [1]}, [3]}, [4]}, the baseline seq2seq model is typically able to generate reasonable pseudotranscripts.
We show that this is not the case with models in our data condition—largely due to domain mismatch—and the standard self-learning approach provides only a minimal gain on seq2seq models.
We present a simple alteration to the standard self-learning paradigm that uses pseudotranscripts generated from a well-trained hybrid system for training a seq2seq model.
This semisupervised technique produces a seq2seq model that is competitive with hybrid systems in the mismatched domain, while hybrid semisupervised models are able to maintain an advantage due to their ability to make better use of LM data.
