We basically follow the definition of no-local operation in (Wang et al., 2018) where f : is a unary function and C : R d × R n×d → R calculates a normalizer, but dispense with the assumption that then the non-local operation degrades to the multihead self-attention as is described in<cite> (Vaswani et al., 2017)</cite> (formula 2 describes only one attention head):