AdamOptimizer[1]} is used
with \(learning\_rate = 0.002, warm\_up = 50000\) , and gradient
clipping at 5.0. Moreover, we employ label smoothing of value \(\epsilon _{ls} = 0.1 \)[2]} and dropout rate of \(P_{drop} = 0.1\) .
For regularization, we apply dropout [3]} in each residual unit of the conformer block and transformer block, i.e, to the output of each module, before it is added to the module input. We set the weight \( \lambda \)  of the CTC branch during joint training to 0.3. During joint decoding, we set the CTC-weight \( \lambda \)  to 0.5. We also train an external n-gram LM followed by WeNet recipe[4]}. To avoid overfitting, we averaged the 30 best model parameters in the development dataset. For the optimal combination of SE-WSBO acoustic model and language model TLG for Aishell-1 testset, the parameters detail as follows: \(acoustic\_scale = 3.26,ngram\_weight=0.93,lattice\_beam=16,beam=32\) .
