Intuitively, the attention mechanism is similar to the human attention
distribution process, that is, in the information processing process,
different content is assigned different attention weights. The essence
of Attention can be described as a query \(Q\)  to a series of key-value
(\(K\) -\(V\) ) pairs of mapping. Self-Attention [1]}
is a special form of Attention Mechanism. The self-attention model
is actually to make \(Q\) , \(K\)  and \(V\)  equal, and do attention inside
the sequence to find the connections within the sequence. Essentially,
for each input vector, Self-Attention generates a vector that is weighted
and summed on its neighboring vectors, where the weight is determined
by the relationship between inputs. For example, if we enter a sentence,
each word in it must be calculated with all the words in the sentence.
The purpose is to learn the word relevance within the sentence and
capture the internal structure of the sentence.
