**<cite>TRANSFORMER</cite>** In this section, we review each component in the standard <cite>transformer model</cite>, and discuss a model structure that is mainly investigated for speech recognition in this work.