The new memory states \(M_{\text{s}}\) and \(z_{\text{s}}\) are then passed to the next segment \(S+1\), building in a recurrence in each attention layer. The right side term \(\sigma(K)^{T}V\) in Eq. (4) is known as an associative binding operator (Smolensky, 1990; Hebb, 2005; Schlag et al., 2020).