The encoder is a bidirectional neural network with gated recurrent units [1]} that reads an input sequence \(x=(x_1,...,x_m)\)  and calculates a forward sequence of hidden states \((\overrightarrow{h}_1,...,\overrightarrow{h}_m)\) ,
and a backward sequence \((\overleftarrow{h}_1,...,\overleftarrow{h}_m)\) .
The hidden states \(\overrightarrow{h}_j\)  and \(\overleftarrow{h}_j\)  are concatenated to obtain the annotation vector \(h_j\) .
