A Survey on Truth Discovery <s> Comparison and summary <s> Although much work in NLP has focused on simply determining what a document means, we also must know whether or not to believe it. Fact-finding algorithms attempt to identify the "truth" among competing claims in a corpus, but fail to take advantage of the user's prior knowledge and presume that truth itself is universal and objective rather than subjective. We introduce a framework for incorporating prior knowledge into any fact-finding algorithm, expressing both general "common-sense" reasoning and specific facts already known to the user as first-order logic and translating this into a tractable linear program. As our results show, this approach scales well to even large problems, both reducing error and allowing the system to determine truth respective to the user rather than the majority. Additionally, we introduce three new fact-finding algorithms capable of outperforming existing fact-finders in many of our experiments. <s> BIB001 </s> A Survey on Truth Discovery <s> Comparison and summary <s> Information retrieval may suggest a document, and information extraction may tell us what it says, but which information sources do we trust and which assertions do we believe when different authors make conflicting claims? Trust algorithms known as fact-finders attempt to answer these questions, but consider only which source makes which claim, ignoring a wealth of background knowledge and contextual detail such as the uncertainty in the information extraction of claims from documents, attributes of the sources, the degree of similarity among claims, and the degree of certainty expressed by the sources. We introduce a new, generalized fact-finding framework able to incorporate this additional information into the fact-finding process. Experiments using several state-of-theart fact-finding algorithms demonstrate that generalized fact-finders achieve significantly better performance than their original variants on both semi-synthetic and real-world problems. <s> BIB002
Here we compare the above three ways to capture the general principle of truth discovery. • In terms of interpretability, iterative methods are easier to understand and interpret. For example, in Investment and PooledInvestment BIB001 BIB002 , sources "invest" their reliabilities among their claimed values and collect credits back from the identified truths. While optimization based and PGM based solutions are derived from coordinate descent and inference, they are interpretable but need more explanations. • When some prior knowledge about sources or objects is available, it can be used to improve the truth discovery performance. In optimization based solutions, prior knowledge can be formulated as extra equality or inequality constraints. For PGM based methods, hyperparameters can capture the external knowledge. All these three ways are widely used to encode the general principle into truth discovery methods. By providing the above comparison, we do not claim that one of them is better than another. In fact, the coordinate descent in optimization based approaches and the parameter inference in PGM based approaches lead to iterative update rules that are similar to iterative methods, and it is possible to formulate the iterative methods as an optimization problem or a parameter inference task. The general procedure of truth discovery is summarized in Algorithm 1. Usually, truth discovery methods start with an initialization of source weights, and then iteratively conduct truth computation step and source weight estimation step. Some stopping criteria are adopted in practice to control the number of iterations. One commonly adopted criterion is to check the change of identified truths or source weights, and terminate the algorithm when the change is smaller than a pre-defined threshold.