Instead of performing a single attention function with \(d\) -dimensional keys, values and queries, [1]} propose to perform Multi-Head Attention, as illustrated in Figure REF  (Middle).
It first linearly projects the queries, keys and values multiple times with different, learned linear projections to \(d\)  dimensions. Then, on each of these projected versions of queries, keys and values, the scaled dot-product attention function is performed in parallel, yielding \(d\) -dimensional output values. These are concatenated and once again projected, resulting in the final values.
\( \text{MultiHead}(\mathbf {Q},\mathbf {K},\mathbf {V}) &= \text{Concat} \left(\text{head}_1,...,\text{head}_H \right) \mathbf {W}^O,\\\text{head}_i &= \text{Attention}(\mathbf {Q}\mathbf {W}_i^Q, \mathbf {K}\mathbf {W}_i^K, \mathbf {V}\mathbf {W}_i^V),\) 
