Multi-turn conversation understanding is a major challenge for building intelligent dialogue systems. This work focuses on retrieval-based response matching for multi-turn conversation whose related work simply concatenates the conversation utterances, ignoring the interactions among previous utterances for context modeling. In this paper, we formulate previous utterances into context using a proposed deep utterance aggregation model to form a fine-grained context representation. In detail, a self-matching attention is first introduced to route the vital information in each utterance. Then the model matches a response with each refined utterance and the final matching score is obtained after attentive turns aggregation. Experimental results show our model outperforms the state-of-the-art methods on three multi-turn conversation benchmarks, including a newly introduced e-commerce dialogue corpus.
With the impressive success of various referential natural language processing studies @cite_32 @cite_14 @cite_2 @cite_22 @cite_21 @cite_18 , developing an intelligent dialogue system becomes realizable, which means training machines to converse with human in natural languages @cite_20 @cite_5 @cite_36 @cite_3 . Towards this end, a number of data-driven dialogue systems are designed @cite_6 @cite_12 @cite_33 @cite_4 @cite_23 @cite_26 , in which modeling multi-turn conversation has drawn more and more attention. To acquire a contextual response, previous utterances are taken as input. concatenated all previous utterances and last utterance as the context representation and then computed the matching degree score based on the context representation to encode candidate response. selected the previous utterances in different strategies and combined them with last utterance to form a reformulated context. performed context-response matching with a multi-view model on both word level and utterance level. improved the leveraging of utterances relationship and contextual information by matching a response with each utterance in the context based on a convolutional neural network.
@cite_18 @cite_14 @cite_4 @cite_22 @cite_33 @cite_26 @cite_36 @cite_21 @cite_32 @cite_3 @cite_6 @cite_23 @cite_2 @cite_5 @cite_12 @cite_20
2122514299 2464349246 2418993857 2954161227
Most previous work on trainable language generation has focused on two paradigms: (a) using a generation decisions of an existing generator. Both approaches rely on the existence of a handcrafted generation component, which is likely to limit their scalability to new domains. The first contribution of this article is to present Bagel, a fully data-driven generation method that treats the language generation task as a search for the most likely sequence of semantic concepts and realization phrases, according to Factored Language Models (FLMs). As domain utterances are not readily available for most natural language generation tasks, a large creative effort is required to produce the data necessary to represent human linguistic variation for nontrivial domains. This article is based on the assumption that learning to produce paraphrases can be facilitated by collecting data from a large sample of untrained annotators using crowdsourcing—rather than a few domain experts—by relying on a coarse meaning representation. A second contribution of this article is to use crowdsourced data to show how dialogue naturalness can be improved by learning to vary the output utterances generated for a given semantic input. Two data-driven methods for generating paraphrases in dialogue are presented: (a) by sampling from the n-best list of realizations produced by Bagel's FLM reranker; and (b) by learning a structured perceptron predicting whether candidate realizations are valid paraphrases. We train Bagel on a set of 1,956 utterances produced by 137 annotators, which covers 10 types of dialogue acts and 128 semantic concepts in a tourist information system for Cambridge. An automated evaluation shows that Bagel outperforms utterance class LM baselines on this domain. A human evaluation of 600 resynthesized dialogue extracts shows that Bagel's FLM output produces utterances comparable to a handcrafted baseline, whereas the perceptron classifier performs worse. Interestingly, human judges find the system sampling from the n-best list to be more natural than a system always returning the first-best utterance. The judges are also more willing to interact with the n-best system in the future. These results suggest that capturing the large variation found in human language using data-driven methods is beneficial for dialogue interaction. Human conversants in dialog adjust their behavior to their conversational partner in many ways. In terms of language use, they adapt to their partners both lexically and syntactically, by using the same referring expressions or sentence structure. In this paper, we describe a natural language generator personage-primed, which can produce utterances entrained to a range of utterance features used in prior utterances by a human user, and represented in the discourse context. Personage-primed can entrain to the user’s referring expressions, tense-modality selection, verb and noun lexical selection, hedge and cue word choice, and syntactic template selection, or any combination of these. To our knowledge, there is no other NLG engine that can dynamically generate all these types of entrainment in any utterance. We report an experiment testing all possible combinations of entrainment in a particular discourse context in order to test whether some types of entrainment are preferred, either because they make the utterance more natural, or because humans perceive the system as more friendly. Our experimental results suggest that human judgements of naturalness are distinct from friendliness: entraining on a user’s hedges increase perceptions of friendliness while reducing naturalness, while entraining on user’s referring expressions, syntactic template selection and tense modal choices increase perceptions of both naturalness and friendliness. We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the high-level coarse tokens, but we argue that a simple extraction procedure is sufficient to capture a wealth of high-level discourse semantics. Such procedure allows training the multiresolution recurrent neural network by maximizing the exact joint log-likelihood over both sequences. In contrast to the standard log- likelihood objective w.r.t. natural language tokens (word perplexity), optimizing the joint log-likelihood biases the model towards modeling high-level abstractions. We apply the proposed model to the task of dialogue response generation in two challenging domains: the Ubuntu technical support domain, and Twitter conversations. On Ubuntu, the model outperforms competing approaches by a substantial margin, achieving state-of-the-art results according to both automatic evaluation metrics and a human evaluation study. On Twitter, the model appears to generate more relevant and on-topic responses according to automatic evaluation metrics. Finally, our experiments demonstrate that the proposed model is more adept at overcoming the sparsity of natural language and is better able to capture long-term structure. We study response selection for multi-turn conversation in retrieval-based chatbots. Existing work either concatenates utterances in context or matches a response with a highly abstract context vector finally, which may lose relationships among utterances or important contextual information. We propose a sequential matching network (SMN) to address both problems. SMN first matches a response with each utterance in the context on multiple levels of granularity, and distills important matching information from each pair as a vector with convolution and pooling operations. The vectors are then accumulated in a chronological order through a recurrent neural network (RNN) which models relationships among utterances. The final matching score is calculated with the hidden states of the RNN. An empirical study on two public data sets shows that SMN can significantly outperform state-of-the-art methods for response selection in multi-turn conversation.