The in-house implementation [1]} of the NMT model proposed by Bahdanau et al. [2]} and Luong et al. [3]} is used as the baseline model; all the other methods were created based on this baseline.
For comparison, settings are unified in all experiments (Table REF ).
In the pre-training process, only the language model is learned; the baseline performs no pre-training, as it does not have access to the language model.
<TABLE><TABLE>