Other than relating <cite>Transformer's</cite> attention mechanism with kernel methods, the prior work (Wang et al., 2018; Shaw et al., 2018; Tsai et al., 2019b ) related the attention mechanism with graph-structured learning.