Neural translation attention mechanism (Bahdanau, Cho & Bengio, 2014) has been shown to be highly beneficial for bi-lingual neural translation of long sentences, but it is not compatible with the multi-task multilingual translation models (Dong et al., 2015;<cite> Luong et al, 2016)</cite> described in the previous Section and character-level translation models (Barzdins & Gosko, 2016) described in this Section.