While the novel attention mechanism, eschewing recurrence, is famous for modeling global dependencies and considered faster than recurrent layers<cite> (Vaswani et al., 2017)</cite> , recent work points out that it may tend to overlook neighboring information (Yang et al., 2019a; Xu et al., 2019) .