Universal language representation is the holy grail in machine translation (MT). Thanks to the new neural MT approach, it seems that there are good perspectives towards this goal. In this paper, we propose a new architecture based on combining variational autoencoders with encoder-decoders and introducing an interlingual loss as an additional training objective. By adding and forcing this interlingual loss, we are able to train multiple encoders and decoders for each language, sharing a common universal representation. Since the final objective of this universal representation is producing close results for similar input sentences (in any language), we propose to evaluate it by encoding the same sentence in two different languages, decoding both latent representations into the same language and comparing both outputs. Preliminary results on the WMT 2017 Turkish English task shows that the proposed architecture is capable of learning a universal language representation and simultaneously training both translation directions with state-of-the-art results.
Classical interlingua approaches @cite_22 aim at finding a universal language that involves a conceptual understanding of all languages over the world. This has been the case of Esperanto @cite_7 or Universal Networking Language @cite_21 and many others. Very differently, in this work, we are focusing on training a universal language representation with deep learning techniques. The objective is to train an intermediate representation that allows to use independent encoders and decoders for each language. Differently, from the classical approach, there is no requirement of semantics for this intermediate representation. Following a similar objective or methodology, most related works are the following ones.
@cite_21 @cite_22 @cite_7
2735935453 2798926775
Presently there has been a surge for representing information in a common language for use cases like question-answering system, machine translation, text summarization etc. UNL has been a centre of attraction for researchers in the past two decades and many have tried to harness its power. This paper throws a glimpse into the strides made by such researchers. As UNL provides a language independent platform it eases out the decision making by accessing the valuable and meaningful information, which is otherwise a challenging errand. Thus it captures information and finds its major application in Natural Language Processing (NLP) domain. The paper is written with the intent to introduce its readers to the UNL framework and explain how it is being used to solve some real world problems. We incorporate an explicit neural interlingua into a multilingual encoder-decoder neural machine translation (NMT) architecture. We demonstrate that our model learns a true interlingua by performing direct zero-shot translation (without using pivot translation), and by using the interlingual sentence embeddings to train an English Yelp review classifier that, through the mediation of the interlingua, can also classify French and German reviews. Furthermore, we show that, despite using a smaller number of parameters than a pairwise collection of bilingual NMT models, our interlingual approach produces comparable BLEU scores for each language pair in WMT15.