and then decoded using a recurrent neural decoder followed by a multi-layer perceptron (MLP). As suggested by Bahdanau et al. [1]}, in order to allow the decoder to attend the entire encoded sequence at every output step, we are using an attention mechanism. At every output step \(k\) , the decoder produces state vectors \(h^d_k\)  and output vectors \(o^d_k\) , based on the previous step context vector \(c_{k-1}\) , the decoder state vector \(h^d_{k-1}\)  and the predicted visual frame index \(y_{k-1}\) . The latter is obtained by an MLP that is applied to the output vector \(o^d_{k-1}\)  and the context vector \(c_{k-1}\) .
