Parallel sentence extraction is a task addressing the data sparsity problem found in multilingual natural language processing applications. We propose a bidirectional recurrent neural network based approach to extract parallel sentences from collections of multilingual texts. Our experiments with noisy parallel corpora show that we can achieve promising results against a competitive baseline by removing the need of specific feature engineering or additional external resources. To justify the utility of our approach, we extract sentence pairs from Wikipedia articles to train machine translation systems and show significant improvements in translation performance.
train a neural machine translation (NMT) system to generate sentence representations with the encoder, which are then used as additional features to the system of Munteanu and Marcu . Similarly, study sentence representations obtained from the encoder of an NMT system to detect new parallel sentence pairs. By comparing cosine similarities, they show that they can distinguish parallel and non-parallel sentences. A different approach exploiting continuous vector representations is proposed in Grover and Mitra . After learning word representations using the bilingual word embeddings model of @cite_27 , they use a convolutional neural network on a similarity matrix to classify if a pair of sentences is aligned or not. These approaches are different from ours, where we use a single end-to-end model to estimate the conditional probability distribution that two sentences are parallel.
@cite_27
2251765408
Recent work in learning bilingual representations tend to tailor towards achieving good performance on bilingual tasks, most often the crosslingual document classification (CLDC) evaluation, but to the detriment of preserving clustering structures of word representations monolingually. In this work, we propose a joint model to learn word representations from scratch that utilizes both the context coocurrence information through the monolingual component and the meaning equivalent signals from the bilingual constraint. Specifically, we extend the recently popular skipgram model to learn high quality bilingual representations efficiently. Our learned embeddings achieve a new state-of-the-art accuracy of 80.3 for the German to English CLDC task and a highly competitive performance of 90.7 for the other classification direction. At the same time, our models outperform best embeddings from past bilingual representation work by a large margin in the monolingual word similarity evaluation. 1