In this work, we use multiple heads to calculate the tensorial self-attention multiple times over the dataset. To this end, we modify the calculation of the multihead attention [1]} to tensorial inputs. Each of these heads correspond to the output of the tensorial self-attention. The outputs are concatenated in the last dimension and each time slice is multiplied with a time slice of a weights tensor with shape \(T \times (H \cdot D) \times F\) , which leads to an output tensor \(Y \in \mathbb {R}^{T \times C \times F}\)  as shown in Eq. (REF ),
\( \begin{aligned}& (Y_{c,f})_{t} = (\operatornamewithlimits{||}_{h=1}^H (Z_h)_{c,h\cdot d})_t \times (W^o_{h\cdot d,f})_t& \forall t=1,\dots ,T,\end{aligned}\) 
