Our starting point for experimentation is a standard baseline neural machine translation system implemented using the Lamtram 1 and DyNet 2 toolkits (Neubig, 2015; Neubig et al., 2017) . Translation models are trained until perplexity convergence on held-out data using the Adam algorithm with a maximum step size of 0.0002 (Kingma and Ba, 2015; <cite>Wu et al., 2016)</cite> .