Following the discussion in Section 2.3, this means that our best performing configuration is conceptually very close to the method by <cite>Faruqui and Dyer (2014)</cite> , as they both coincide on maximizing the average dimension-wise covariance and length-normalize the embeddings in both languages first, the only difference being that our model enforces monolingual invariance after the normalization while theirs does change the monolingual embeddings to make different dimensions have the same variance and be uncorrelated among themselves.