A Survey on Truth Discovery <s> Source dependency analysis <s> Many data management applications, such as setting up Web portals, managing enterprise data, managing community data, and sharing scientific data, require integrating data from multiple sources. Each of these sources provides a set of values and different sources can often provide conflicting values. To present quality data to users, it is critical that data integration systems can resolve conflicts and discover true values. Typically, we expect a true value to be provided by more sources than any particular false one, so we can take the value provided by the majority of the sources as the truth. Unfortunately, a false value can be spread through copying and that makes truth discovery extremely tricky. In this paper, we consider how to find true values from conflicting information when there are a large number of sources, among which some may copy from others. ::: ::: We present a novel approach that considers dependence between data sources in truth discovery. Intuitively, if two data sources provide a large number of common values and many of these values are rarely provided by other sources (e.g., particular false values), it is very likely that one copies from the other. We apply Bayesian analysis to decide dependence between sources and design an algorithm that iteratively detects dependence and discovers truth from conflicting information. We also extend our model by considering accuracy of data sources and similarity between values. Our experiments on synthetic data as well as real-world data show that our algorithm can significantly improve accuracy of truth discovery and is scalable when there are a large number of data sources. <s> BIB001 </s> A Survey on Truth Discovery <s> Source dependency analysis <s> Modern information management applications often require integrating data from a variety of data sources, some of which may copy or buy data from other sources. When these data sources model a dynamically changing world (e.g., people's contact information changes over time, restaurants open and go out of business), sources often provide out-of-date data. Errors can also creep into data when sources are updated often. Given out-of-date and erroneous data provided by different, possibly dependent, sources, it is challenging for data integration systems to provide the true values. Straightforward ways to resolve such inconsistencies (e.g., voting) may lead to noisy results, often with detrimental consequences. ::: ::: In this paper, we study the problem of finding true values and determining the copying relationship between sources, when the update history of the sources is known. We model the quality of sources over time by their coverage, exactness and freshness. Based on these measures, we conduct a probabilistic analysis. First, we develop a Hidden Markov Model that decides whether a source is a copier of another source and identifies the specific moments at which it copies. Second, we develop a Bayesian model that aggregates information from the sources to decide the true value for a data item, and the evolution of the true values over time. Experimental results on both real-world and synthetic data show high accuracy and scalability of our techniques. <s> BIB002 </s> A Survey on Truth Discovery <s> Source dependency analysis <s> Web technologies have enabled data sharing between sources but also simplified copying (and often publishing without proper attribution). The copying relationships can be complex: some sources copy from multiple sources on different subsets of data; some co-copy from the same source, and some transitively copy from another. Understanding such copying relationships is desirable both for business purposes and for improving many key components in data integration, such as resolving conflicts across various sources, reconciling distinct references to the same real-world entity, and efficiently answering queries over multiple sources. Recent works have studied how to detect copying between a pair of sources, but the techniques can fall short in the presence of complex copying relationships. ::: ::: In this paper we describe techniques that discover global copying relationships between a set of structured sources. Towards this goal we make two contributions. First, we propose a global detection algorithm that identifies co-copying and transitive copying, returning only source pairs with direct copying. Second, global detection requires accurate decisions on copying direction; we significantly improve over previous techniques on this by considering various types of evidence for copying and correlation of copying on different data items. Experimental results on real-world data and synthetic data show high effectiveness and efficiency of our techniques. <s> BIB003 </s> A Survey on Truth Discovery <s> Source dependency analysis <s> Collective intelligence, which aggregates the shared information from large crowds, is often negatively impacted by unreliable information sources with the low quality data. This becomes a barrier to the effective use of collective intelligence in a variety of applications. In order to address this issue, we propose a probabilistic model to jointly assess the reliability of sources and find the true data. We observe that different sources are often not independent of each other. Instead, sources are prone to be mutually influenced, which makes them dependent when sharing information with each other. High dependency between sources makes collective intelligence vulnerable to the overuse of redundant (and possibly incorrect) information from the dependent sources. Thus, we reveal the latent group structure among dependent sources, and aggregate the information at the group level rather than from individual sources directly. This can prevent the collective intelligence from being inappropriately dominated by dependent sources. We will also explicitly reveal the reliability of groups, and minimize the negative impacts of unreliable groups. Experimental results on real-world data sets show the effectiveness of the proposed approach with respect to existing algorithms. <s> BIB004 </s> A Survey on Truth Discovery <s> Source dependency analysis <s> Many applications rely on Web data and extraction systems to accomplish knowledge-driven tasks. Web information is not curated, so many sources provide inaccurate, or conflicting information. Moreover, extraction systems introduce additional noise to the data. We wish to automatically distinguish correct data and erroneous data for creating a cleaner set of integrated data. Previous work has shown that a naive voting strategy that trusts data provided by the majority or at least a certain number of sources may not work well in the presence of copying between the sources. However, correlation between sources can be much broader than copying: sources may provide data from complementary domains (negative correlation), extractors may focus on different types of information (negative correlation), and extractors may apply common rules in extraction (positive correlation, without copying). In this paper we present novel techniques modeling correlations between sources and applying it in truth finding. We provide a comprehensive evaluation of our approach on three real-world datasets with different characteristics, as well as on synthetic data, showing that our algorithms outperform the existing state-of-the-art techniques. <s> BIB005 </s> A Survey on Truth Discovery <s> Source dependency analysis <s> The explosive growth in social network content suggests that the largest "sensor network" yet might be human. Extending the participatory sensing model, this paper explores the prospect of utilizing social networks as sensor networks, which gives rise to an interesting reliable sensing problem. In this problem, individuals are represented by sensors (data sources) who occasionally make observations about the physical world. These observations may be true or false, and hence are viewed as binary claims. The reliable sensing problem is to determine the correctness of reported observations. From a networked sensing standpoint, what makes this sensing problem formulation different is that, in the case of human participants, not only is the reliability of sources usually unknown but also the original data provenance may be uncertain. Individuals may report observations made by others as their own. The contribution of this paper lies in developing a model that considers the impact of such information sharing on the analytical foundations of reliable sensing, and embed it into a tool called Apollo that uses Twitter as a "sensor network" for observing events in the physical world. Evaluation, using Twitter-based case-studies, shows good correspondence between observations deemed correct by Apollo and ground truth. <s> BIB006
In [14-16, 49, 50, 53] , the authors relax source independence assumption. In these work, the authors try to detect copying relationship among sources, and then adjust the sources' weights according to the detected relationship. This copying detection is beneficial in application scenarios where some sources copy information from others. The main principle behind the copy detection is that if some sources make many common mistakes, they are not likely to be independent with each other. However, this principle becomes ineffective when some sources copy information from a good source. In BIB001 , the authors propose a method to detect direct copying relationship. They assume that a copier does not copy all the information from other sources and may provide some information by itself. In other words, the detected copying relationship is represented by a probability that a source copies the provided information from others. The authors apply Bayesian analysis to infer the existence and direction of copying relationship among sources. This copying detection procedure is tightly combined with truth discovery, and thus the detected copying relationship and the discovered truths are iteratively updated. On the other hand, BIB004 alleviates the source dependency problem by revealing the latent group structure among sources, and aggregates the information at the group level. This can reduce the risk of overusing the information from the dependent sources, especially when these sources are unreliable. Compared with the above work that the copying relationship is detected from snapshots of data, BIB002 detects copying relationship from dynamic data, where the provided information from sources are changing. The authors apply Hidden Markov Model (HMM) to detect copying relationship given the update history of sources, and the outputs are the evolving copying relationship and the evolving true values. To evaluate the quality of sources, instead of simply considering their accuracy, more metrics are adopted for the dynamic data, such as coverage (how many values in the history a source covers), exactness (how many updates conform to the reality), and freshness (how quickly a source captures a new value). The copying relationship detection is performed by considering the source quality and the update behaviors among sources simultaneously. Similar to BIB001 , both the copying relationship and true values are inferred in an iterative procedure. However, there may be other complex copying relationships rather than direct copying, such as co-copying (i.e., multiple sources copy from one source), and transitive copying (i.e., a source may copy from other sources transitively). To detect such global copying relationship, BIB003 extends the idea of BIB001 and BIB002 : By considering both the completeness and accuracy of sources, the existence and direction of direct copying relationship are detected. Then based on these detected direct copying relationship, Bayesian analysis is performed to detect global copying relationship. The main principle behind the copy detection is that copiers make the same mistakes with sources they copy from, which is limited in characterizing the relations among sources. In BIB005 , the authors consider different correlations between sources which are more general than copying. Sources may have positive correlations, such as copying information from each other or sharing similar rules for information extractors; sources may also have negative correlations, such as providing data from complementary domains or focusing on different fields of information. Intuitively, the observations from positively correlated sources should not increase the belief that they are true, and the observations supported by only a few negatively correlated sources should not decrease the belief that they are true. The proposed method models correlations among sources with joint precision and joint recall, and Bayesian analysis is preformed to infer the probability of an observation to be true. Approximation methods are also provided as the computation complexity of estimating joint precision and recall is exponential. In above work, source correlations are inferred from data. While in practice, such source correlation information might be available as extra input. For example, the authors in BIB006 fuse the observations from multiple Twitter users by estimating the reliability of each individual user (source). In this application scenario, Twitter users may report observations made by others as their own, which is a form of source copying phenomenon. Instead of estimating the correlations among users, the follower-followee graph and retweeting behaviors in Twitter capture the correlations among users, and such extra information can be adopted to improve the results of truth discovery algorithms.