For that purpose, state-of-the-art approaches rely on either a separately trained unsupervised Statistical Machine Translation (SMT) system, which is used for warmup during the initial back-translation iterations (Marie and Fujita, 2018;<cite> Artetxe et al., 2019)</cite> , or large-scale pre-training through masked denoising, which is used to initialize the weights of the underlying encoder-decoder (Conneau and Lample, 2019; Song et al., 2019; Liu et al., 2020) .