We compare the context-unaware method (i.e., directly calculating the similarity scores between unsupervised cross-lingual embeddings (Artetxe et al., 2018a ) of source and target words), XLM <cite>(Lample and Conneau, 2019)</cite> and our proposed CMLM pre-training method in the Table 3 .