Benefiting from the availability of large-scale benchmark datasets such as SQuAD (Rajpurkar et al., 2016) , the attention-based neural networks has spread to machine comprehension and question answering tasks to allow the model to attend over past output vectors (Wang & Jiang, 2017; Seo et al., 2017; Xiong et al., 2017;<cite> Hu et al., 2017</cite>; Pan et al., 2017) .