We use a bi-directional long short-term memory (Bi-LSTM) [1]} layer to capture the interaction among words in a sentence \(S=\lbrace w_1, w_2, ....., w_n\rbrace \) , where \(n\)  is the sentence length. The input to this layer is the concatenated vector \(\mathbf {x} \in \mathbb {R}^{d_w + d_z}\)  of word embedding vector \(\mathbf {w}\)  and entity token indicator embedding vector \(\mathbf {z}\) .
\(&\mathbf {x}_t=\mathbf {w}_t ~||~ \mathbf {z}_t \nonumber \\&\overrightarrow{\mathbf {h}_t} = \overrightarrow{\mathrm {LSTM}}(\mathbf {x}_t, \mathbf {h}_{t-1}) \nonumber \\&\overleftarrow{\mathbf {h}_t} = \overleftarrow{\mathrm {LSTM}}(\mathbf {x}_t, \mathbf {h}_{t+1}) \nonumber \\&\mathbf {h}_t = \overrightarrow{\mathbf {h}_t} || \overleftarrow{\mathbf {h}_t}\) 
