In [16, 17, 18] , self-attention is only applied in a chunk of acoustic input restricted by a time window, which makes the <cite>transformer model</cite> easier to train as it does not need to consider very long term correlations.