Vaswani et al. were the first to apply attention directly over the word-embeddings, and thus derived a new neural network architecture which, without any recurrence, achieved state-ofthe-art results in machine translation <cite>[2]</cite> .