Effective Cross lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
Transfer learning or multilingual model is essential for low resource neural machine translation (NMT) but the applicability is limited to cognate languages by sharing their vocabularies. This paper shows effective techniques to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies. We relieve the vocabulary mismatch by using cross lingual word embedding, train a more language agnostic encoder by injecting artificial noises, and generate synthetic data easily from the pretraining data without back translation. Our methods do not require restructuring the vocabulary or retraining the model. We improve plain NMT transfer by up to +5.1% BLEU in five low resource translation tasks, outperforming multilingual joint training by a large margin. We also provide extensive ablation studies on pretrained embedding, synthetic data, vocabulary size, and parameter freezing for a better understanding of NMT transfer.
ACL