Basic Module  We implement the proposed method upon the Multi-Head Attention (MHA) and Feed-Forward Network (FFN) [1]}. The MHA consists of \(n\)  parallel heads and each head is defined as a scaled dot-product attention:
\(&\text{Att}_i(X,Y) = \text{softmax}\left(\frac{X\text{W}_i^\text{Q}(Y\text{W}_i^\text{K})^T}{\sqrt{{d}_{n}}}\right)Y\text{W}_i^\text{V} \nonumber \\& \text{MHA}(X,Y) = [\text{Att}_1(X,Y); \dots ; \text{Att}_n(X,Y)]\text{W}^\text{O}\) 
