For future work, it would be interesting to investigate a data condition with a similar mismatch where significantly more out-of-domain transcribed data was available.
It would also be interesting to investigate domains that are not easily captured through text.
Our current seq2seq models are fairly simple, so we will investigate more powerful model architectures such as the Conformer [1]}, as well as additional types of external LMs for seq2seq models.
We also plan to explore unsupervised pretraining [2]}, especially as it has been shown to be complementary to self-learning [3]}.
