Neural conversational models are widely used in applications like personal assistants and chat bots. These models seem to give better performance when operating on word level. However, for fusion languages like French, Russian and Polish vocabulary size sometimes become infeasible since most of the words have lots of word forms. We propose a neural network architecture for transforming normalized text into a grammatically correct one. Our model efficiently employs correspondence between normalized and target words and significantly outperforms character-level models while being 2x faster in training and 20\% faster at evaluation. We also propose a new pipeline for building conversational models: first generate a normalized answer and then transform it into a grammatically correct one using our network. The proposed pipeline gives better performance than character-level conversational models according to assessor testing. Neural conversational models BID18 are used in a large number of applications: from technical support and chat bots to personal assistants. While being a powerful framework, they often suffer from high computational costs.The main computational and memory bottleneck occurs at the vocabulary part of the model. Vocabulary is used to map a sequence of input tokens to embedding vectors: one embedding vector is stored for each word in vocabulary.English is de-facto a standard language for training conversational models, mostly for a large number of speakers and simple grammar. In english, words usually have only a few word forms. For example, verbs may occur in present and past tenses, nouns can have singular and plural forms.For many other languages, however, some words may have tens of word forms. This is the case for Polish, Russian, French and many other languages. For these languages storing all forms of frequent words in a vocabulary significantly increase computational costs.To reduce vocabulary size, we propose to normalize input and output sentences by putting them into a standard form. Generated texts can then be converted into grammatically correct ones by solving morphological agreement task. This can be efficiently done by a model proposed in this work.Our contribution is two-fold:• We propose a neural network architecture for performing morphological agreement in fusion languages such as French, Polish and Russian (Section 2).• We introduce a new approach to building conversational models: generating normalized text and then performing morphological agreement with proposed model (Section 3); In this paper we proposed a neural network model that can efficiently employ relationship between input and output words in morphological agreement task. We also proposed a modification for this model that uses context sentence. We apply this model for neural conversational model in a new pipeline: we use normalized question to generate normalized answer and then apply proposed model to obtain grammatically correct response. This model showed better performance than character level neural conversational model based on assessors responses.We achieved significant improvement comparing to character-level, bigram and hierarchical sequenceto-sequence models on morphological agreement task for Russian, French and Polish languages. Trained models seem to understand main grammatical rules and notions such as tenses, cases and pluralities.
Proposed architecture to solve morphological agreement task
{"src": "scitldr-AIC_train", "paper_id": "HyTrSegCb"}