To sustain engaging conversation, it is critical for chatbots to make good use of relevant knowledge. Equipped with a knowledge base, chatbots are able to extract conversation-related attributes and entities to facilitate context modeling and response generation. In this work, we distinguish the uses of attribute and entity and incorporate them into the encoder-decoder architecture in different manners. Based on the augmented architecture, our chatbot, namely Mike, is able to generate responses by referring to proper entities from the collected knowledge. To validate the proposed approach, we build a movie conversation corpus on which the proposed approach significantly outperforms other four knowledge-grounded models.
However, the symbolic nature of KGs impedes their applications. To tackle this issue, knowledge graph embedding models have been proposed to embed the relations and entities in a KG into low-dimensional continuous vector spaces. These KG embedding models can be roughly categorized into two groups: translation-based models and semantic matching models. Specifically, translation-based models learn the embeddings by calculating the plausibility of a fact as the distance between the two entities, usually after a translation carried out by the relation. Representative models are TransE @cite_5 , TransH @cite_35 , TransR @cite_45 . In TransE @cite_5 , the entity and relation embedding vectors are in the same space. In TransH @cite_35 , entity embedding vectors are projected into a relation-specific hyperplane. In TransR @cite_45 , entities are projected from the entity space to the relation space. kgsurvey summarizes other advanced knowledge embedding approaches. In this work, we embed our KG using the widely-adopted TransE model @cite_5 , and integrate the knowledge embeddings into conversation models in a novel way.
@cite_5 @cite_45 @cite_35
2127795553 2283196293
We consider the problem of embedding entities and relationships of multi-relational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose TransE, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities. Despite its simplicity, this assumption proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. Besides, it can be successfully trained on a large scale data set with 1M entities, 25k relationships and more than 17M training samples. We deal with embedding a large scale knowledge graph composed of entities and relations into a continuous vector space. TransE is a promising method proposed recently, which is very efficient while achieving state-of-the-art predictive performance. We discuss some mapping properties of relations which should be considered in embedding, such as reflexive, one-to-many, many-to-one, and many-to-many. We note that TransE does not do well in dealing with these properties. Some complex models are capable of preserving these mapping properties but sacrifice efficiency in the process. To make a good trade-off between model capacity and efficiency, in this paper we propose TransH which models a relation as a hyperplane together with a translation operation on it. In this way, we can well preserve the above mapping properties of relations with almost the same model complexity of TransE. Additionally, as a practical knowledge graph is often far from completed, how to construct negative examples to reduce false negative labels in training is very important. Utilizing the one-to-many many-to-one mapping property of a relation, we propose a simple trick to reduce the possibility of false negative labeling. We conduct extensive experiments on link prediction, triplet classification and fact extraction on benchmark datasets like WordNet and Freebase. Experiments show TransH delivers significant improvements over TransE on predictive accuracy with comparable capability to scale up.