
The interleaved structure makes the architecture of encoder and decoder layers consistent [1]}, facilitating shared parameterization of attention modules throughout the encoder and decoder.

As shown in Table REF , the lightweight FFNs that interleave attention modules in the decoder reduce FLOPS and save a large number of parameters for decoder FFNs' parameterization that is very uneconomical.

