In this paper, we have presented a <cite>transformer</cite> model with interleaved self-attention and convolution for hybrid acoustic modeling, although this structure may be also applicable to end-to-end models.