We compared the ASR performance of the proposed method on WER metrics with three methods.
First two methods were Conformer(L), and Conformer(L)+ which has 19 Conformer blocks and whose model size is roughly the same as the proposed method, respectively. These models were trained for 300k steps from scratch. These models gives us the baseline WER of using large scale ASR models without SE frontend.
The third model jointly trains the supervised SE frontend and Conformer(L) ASR model where the network architecture of the SE frontend was DF-Conformer-8 used in Sec, REF . This model uses \(\mathcal {L}^{\mbox{\tiny {ASR}}} + \gamma \mathcal {L}^{\mbox{\tiny {SE}}}\)  where \(\gamma = 0.25\)  was determined based on WER on the validation dataset. In addition, we skipped the SE frontend with 50% probability in fine-tunining stage as a multi-condition learning strategy [1]}. These models gives us WER of the standard joint training strategies of an SE frontend and a large scale ASR backend.
