The LSTMs are initialized using semi-orthogonal matrices as suggested by Saxe et al. [1]}, the convolutional and linear layers were initialized using Xavier Normal [2]} and the batch normalization layers were initialized with a normal distribution (\(1 \pm 0.02\) ). We use an Adam optimizer [3]} with a learning rate of \(10^{-6}\)  and beta coefficients of 0.5 and 0.999. At train time, we apply dropout with 0.1 probability on the LSTM decoder's previous-step output-embeddings.
