In the original architecture (<cite> [4]</cite> ), attention follows the last recurrent layer. To have more insight on the representation learnt by the network, we added an attention mechanism after the first recurrent layer.