Neural machine translation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2015) is rapidly proving itself to be a strong competitor to other statistical machine translation methods. However, it still lags behind other statistical methods on very lowresource language pairs<cite> (Zoph et al., 2016</cite>; Koehn and Knowles, 2017) .