The attention we use here is similar to the formulation used in<cite> (Hill et al., 2015</cite>; Sukhbaatar et al., 2015) , but with two differences. First, we use a bilinear term instead of a simple dot product in order to compute the importance of each query term in the current time step. This simple bilinear attention has been successfully used in (Luong et al., 2015) . Second, we add a term a q that allows to bias the attention mechanism towards words which tend to be important across the questions independently of the search key s t−1 .