Bilingual Distributed Word Representations from Document Aligned Comparable Data
We propose a new model for learning bilingual word representations from nonparallel document aligned data. Following the recent advances in word representation learning, our model learns dense real valued word vectors, that is, bilingual word embeddings (BWEs) Unlike prior work on inducing BWEs which heavily relied on parallel sentence aligned corpora and/or readily available translation resources such as dictionaries, the article reveals that BWEs may be learned solely on the basis of document aligned comparable data without any additional lexical resources nor syntactic information. We present a comparison of our approach with previous state of the art models for learning bilingual word representations from comparable data that rely on the framework of multilingual probabilistic topic modeling (MuPTM) as well as with distributional local context counting models. We demonstrate the utility of the induced BWEs in two semantic tasks: (1) bilingual lexicon extraction, (2) suggesting word translations in context for polysemous words. Our simple yet effective BWE based models significantly outperform the MuPTM based and context counting representation models from comparable data as well as prior BWE based models, and acquire the best reported results on both tasks for all three tested language pairs.
J. Artif. Intell. Res.