In particular, we extend the model of<cite> Conneau et al. (2018)</cite> by using a cycle consistency loss (Zhou et al., 2016) to regularize two models in opposite directions. Our model significantly outperforms competitive baselines, obtaining the best published results.