Different from previously known coattention mechanisms such as (Xiong et al., 2017; Lu et al., 2016a) , our co-attention is built through connecting two multiplicative attention modules<cite> (Vaswani et al., 2017</cite> ) each containing three gates, i.e., Value, Key and Query.