Attention mechanisms in neural networks have been successfully applied to a variety of problems, like machine translation [Bahdanau et al., 2014] , image captioning [Xu et al., 2015] , supervised relation extraction [Shen and Huang, 2016] , distantly-supervised relation extraction<cite> [Zheng et al., 2016]</cite> etc.