Attention has emerged as a prominent neural module extensively adopted in a wide range of deep learning research problems (Das et al., 2017; Rocktäschel et al., 2015; Santos et al., 2016; Xu and Saenko, 2016; Yang et al., 2016; Yin et al., 2016; Zhu et al., 2016; Xu et al., 2015; Chorowski et al., 2015) such as VQA, reading comprehension, textual entailment, image captioning, speech recognition and so forth. It's remarkable success is also embodied in machine translation tasks (Bahdanau et al., 2014;<cite> Vaswani et al., 2017)</cite> .