In our work, we propose a self-learning framework to incorporate the predictions of the multilingual BERT model<cite> (Devlin et al., 2018)</cite> on non-English data into an English training procedure.