We build the baseline model with a 12-layer encoder and
a 6-layer decoder following the WeNet recipe[1]}.
We employ h = 4 parallel attention heads in the blockformer models. For every layer, we use \(d_{k}=d_{v}=d_{model}/h=64\) , \(d_{ffn}= 2048\)  . The base model has about 46M parameters. Our Base-WSOLB model only adds some scaler weight( Increased number of parameters: 18 ), SE-WSBO model adds SE-module( Increased number of parameters: 360 ).
