1. Self-Attention (SA): The SA module [1]} is used in the Transformer architecture to focus on significant parts of a given input by capturing the interaction between different vectors in \(\mathbf {Z}\in \mathbb {R}^{N\times d}\) , where \(\mathbf {Z}\)  consists of \(N\)  vectors, each with an embedding dimension of \(d\) . Towards this goal, three different matrices are defined, named Queries \(\mathbf {Q}\) , Keys \(\mathbf {K}\) , and Values \(\mathbf {V}\) , computed by a linear transformation as follows
\([\mathbf {Q}, \mathbf {K}, \mathbf {V}] = \mathbf {Z}\mathit {W}^{QKV},\) 
