In the next step, each time step \(t\)  slice of \(Q\)  (notated as \((Q_{c,d})_t\) ) is multiplied with each time step \(t^{\prime }\)  transposed slice of \(K\)  (notated as \(((K_{c^{\prime },d})_{t^{\prime }})^T\) ). All multiplications result in matrices of shape \(C \times C^{\prime }\) , which are slices of a tensor \(\tilde{R} \in \mathbb {R}^{T \times T^{\prime } \times C \times C^{\prime }}\) , where \(T\)  and \(C\)  are the first and second dimensions of \(Q\)  tensor and \(T^{\prime }\)  and \(C^{\prime }\)  are the first and second dimensions of \(K\)  tensor. \(\tilde{R}\)  tensor is in turn summed over the last dimension and divided element-wise by the square root of \(D\)  (following the formulation in [1]}) producing a tensor \(R \in \mathbb {R}^{T \times T^{\prime } \times C}\) . The entire calculation is shown in Eq. (REF ).
\( \begin{aligned}& (\tilde{R}_{c,c^{\prime }})_{t,t^{\prime }} = (Q_{c,d})_{t} \times ((K_{c^{\prime },d})_{t^{\prime }})^T \;\;\; \forall t,t^{\prime }=1,\dots ,T, \\& R = \frac{1}{\sqrt{D}} \sum _{c^{\prime }=1}^{C} (\tilde{R}_{t,t^{\prime },c})_{c^{\prime }}.\\\end{aligned}\) 
