Similar to masked language models for contextual word representations (Devlin et al., 2019; Liu et al., 2019 ), a con- ditional masked language model (CMLM, <cite>Ghazvininejad et al. (2019)</cite> ) predicts randomly masked target tokens Y mask given a source text X and the rest of the target tokens Y obs .