Finally, for the H1-seeded models, we can add the Set 2 LM data to observe the effect of additional text data not seen in pseudotranscription.
The hybrid model gets an improvement of 2.8% WER on average compared to the 1.0% WER gain from the seq2seq model, again showing the power that additional LM data can provide hybrid models that our current seq2seq models are unable to replicate.
Perhaps other approaches beyond shallow fusion like deep [1]} and cold fusion [2]} would provide further gains from the additional LM data, but previous work has seen limited gains from these techniques [3]}, [4]}.
