Populating Knowledge Base ( KB ) with new knowledge facts from reliable text resources usually consists of linking name mentions to KB entities and identifying relationship between entity pairs . However , the task often suffers from errors propagating from upstream entity linkers to downstream relation extractors . In this paper , we propose a novel joint inference framework to allow interactions between the two subtasks and find an optimal assignment by addressing the coherence among preliminary local predictions : whether the types of entities meet the expectations of relations explicitly or implicitly , and whether the local predictions are globally compatible . We further measure the confidence of the extracted triples by looking at the details of the complete extraction process . Experiments show that the proposed framework can significantly reduce the error propagations thus obtain more reliable facts , and outperforms competitive baselines with state-of-the-art relation extraction models . Recent advances in natural language processing have made it possible to construct structured KBs from online encyclopedia resources , at an unprecedented scale and much more efficiently than traditional manual edit . However , in those KBs , entities which are popular to the community usually contain more knowledge facts , e.g. , the basketball player LeBron James , the actor Nicholas Cage , etc . , while most other entities often have fewer facts . On the other hand , knowledge facts should be updated as the development of entities , such as changes in the cabinet , a marriage event , or an acquisition between two companies , etc . In order to address the above issues , we could consult populating existing KBs from reliable text resources , e.g. , newswire , which usually involves enriching KBs with new entities and populating KBs with new knowledge facts , in the form of < Entity , Relation , Entity > triple . In this paper , we will focus on the latter , identifying relationship between two existing KB entities . This task can be intuitively considered in a pipeline paradigm , that is , name mentions in the texts are first linked to entities in the KB ( entity linking , EL ) , and then the relationship between them are identified ( relation extraction , RE ) . It is worth mentioning that the first task EL is different from the task of named entity recognition ( NER ) in traditional information extraction ( IE ) tasks , where NER recognizes and classifies the entity mentions ( to several predefined types ) in the texts , but EL focuses on linking the mentions to their corresponding entities in the KB . Such pipeline systems often suffer from errors propagating from upstream to downstream , since only the local best results are selected to the next step . One idea to solve the problem is to allow interactions among the local predictions of both subtasks and jointly select an optimal assignment to eliminate possible errors in the pipeline . Let us first look at an example . Suppose we are extracting knowledge facts from two sentences in Figure 1 : in sentence [ 1 ] , if we are more confident to extract the relation fb : org.headquarters1 , we will be then prompted to select Bryant University , which indeed favors the RE prediction that requires an organization to be its subject . On the other side , if we are sure to link to Kobe Bryant in sentence [ 2 ] , we will probably select fb : pro athlete.teams , whose subject position expects an athlete , e.g. , an NBA player . It is not difficult to see that the argument type expectations of relations can encourage the two subtasks interact with each other and select coherent predictions for both of them . In KBs with well-defined schemas , such as Freebase , type requirements can be collected and utilized explicitly ( Yao et al . , 2010 ) . However , in other KBs with less reliable or even no schemas , it is more appropriate to implicitly capture the type expectations for a given relation ( Riedel et al . , 2013 ) . Furthermore , previous RE approaches usually process each triple individually , which ignores whether those local predictions are compatible with each other . For example , suppose the local predictions of the two sentences above are < Kobe Bryant , fb : org.headquarters , Smithfield , Rhode Is-land > and < Kobe Bryant , fb : pro athlete.teams , Los Angeles Lakers > , respectively , which , in fact , disagree with each other with respect to the KB , since , in most cases , these two relations can not share subjects . Now we can see that either the relation predictions or the EL results for " Bryant " are incorrect . Those disagreements provide us an effective way to remove the possible incorrect predictions that cause the incompatibilities . On the other hand , the automatically extracted knowledge facts inevitably contain errors , especially for those triples collected from open domain . Extractions with confidence scores will be more than useful for users to make proper decisions according to their requirements , such as trading recall for precision , or supporting approximate queries . In this paper , we propose a joint framework to populate an existing KB with new knowledge facts extracted from reliable text resources . The joint framework is designed to address the error propagation issue in a pipeline system , where subtasks are optimized in isolation and locally . We find an optimal configuration from top k results of both subtasks , which maximizes the scores of each step , fulfills the argument type expectations of relations , which can be captured explicitly or implicitly , in the KB , and avoids globally incoherent predictions . We formulate this optimization problem in an Integer Linear Program ( ILP ) framework , and further adopt a logistic regression model to measure the reliability of the whole process , and assign confidences to all extracted triples to facilitate further applications . The experiments on a real-world case study show that our framework can eliminate error propagations in the pipeline systems by taking relations ' argument type expectations and global compatibilities into account , thus outperforms the pipeline approaches based on state-ofthe-art relation extractors by a large margin . Furthermore , we investigate both explicit and implicit type clues for relations , and provide suggestions about which to choose according to the characteristics of existing KBs . Additionally , our proposed confidence estimations can help to achieve a precision of over 85 % for a considerable amount of high quality extractions . In the rest of the paper , we first review related work and then define the knowledge base population task that we will address in this paper . Next we detail the proposed framework and present our experiments and results . Finally , we conclude this paper with future directions . In this paper , we propose a joint framework for the task of populating KBs with new knowledge facts , which performs joint inference on two subtasks , maximizes their preliminary scores , fulfills the type expectations of relations and avoids global incompatibilities with respect to all local predictions to find an optimal assignment . Experimental results show that our framework can significantly eliminate the error propagations in pipeline systems and outperforms competitive pipeline systems with state-of-the-art RE models . Regarding the explicit argument type expectations and the implicit ones , the latter can result in a higher overall precision , while the former performs better in acquiring high quality knowledge facts with higher confidence control , indicating that if the KB has a well-defined schema we can use explicit type requirements for the KBP task , and if not , our model can still perform well by mining the implicit ones . Our framework can also generalize well with other preliminary RE models . Furthermore , we assign extraction confidences to all extracted facts to facilitate further applications . By setting a suitable threshold , our framework can populate high quality reliable knowledge facts to existing KBs . For future work , we will address the NIL issue of EL where we currently assume all entities should be linked to a KB . It would be also interesting to jointly model the two subtasks through structured learning , instead of joint inference only . Currently we only use the coherence of extracted triples and the KB to estimate confidences , which would be nice to directly model the issue in a joint model .
Existing pipeline approaches to populate Knowledge Base with new knowledge facts from texts suffer from error propagating from upstream entity linkers to downstream relation extractors.
They propose to formulate the problem in an Integer Linear Program to find an optimal configuration from the top k results of both tasks.
They show that the proposed framework can reduce error propagations and outperform competitive pipeline baselines with state-of-the-art relation extraction models.