A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> For many KDD applications, such as detecting criminal activities in E-commerce, finding the rare instances or the outliers, can be more interesting than finding the common patterns. Existing work in outlier detection regards being an outlier as a binary property. In this paper, we contend that for many scenarios, it is more meaningful to assign to each object a degree of being an outlier. This degree is called the local outlier factor (LOF) of an object. It is local in that the degree depends on how isolated the object is with respect to the surrounding neighborhood. We give a detailed formal analysis showing that LOF enjoys many desirable properties. Using real-world datasets, we demonstrate that LOF can be used to find outliers which appear to be meaningful, but can otherwise not be identified with existing approaches. Finally, a careful performance evaluation of our algorithm confirms we show that our approach of finding local outliers can be practical. <s> BIB001 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Abstract This paper presents a general, rigorous, and fault-tolerant framework for maintaining consistent mean and covariance estimates in an arbitrary, dynamic, distributed network of information processing nodes. In particular, a solution is provided that addresses the information deconfliction problem that arises when estimates from two or more different nodes are determined to be inconsistent with each other, e.g., when two high precision (small covariance) estimates place the position of a particular object at very different locations. The challenge is to be able to resolve such inconsistencies without having to access and exploit global information to determine which of the estimates is spurious. The solution proposed in this paper is called Covariance Union. <s> BIB002 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Spatial outliers are the spatial objects with distinct features from their surrounding neighbors. Detection of spatial outliers helps reveal valuable information from large spatial data sets. In many real applications, spatial objects can not be simply abstracted as isolated points. They have different boundary, size, volume, and location. These spatial properties affect the impact of a spatial object on its neighbors and should be taken into consideration. In this paper, we propose two spatial outlier detection methods which integrate the impact of spatial properties to the outlierness measurement. Experimental results on a real data set demonstrate the effectiveness of the proposed algorithms. <s> BIB003 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Modern machine learning-based approaches to computer vision require very large databases of hand labeled images. Some contemporary vision systems already require on the order of millions of images for training (e.g., Omron face detector [9]). New Internet-based services allow for a large number of labelers to collaborate around the world at very low cost. However, using these services brings interesting theoretical and practical challenges: (1) The labelers may have wide ranging levels of expertise which are unknown a priori, and in some cases may be adversarial; (2) images may vary in their level of difficulty; and (3) multiple labels for the same image must be combined to provide an estimate of the actual label of the image. Probabilistic approaches provide a principled way to approach these problems. In this paper we present a probabilistic model and use it to simultaneously infer the label of each image, the expertise of each labeler, and the difficulty of each image. On both simulated and real data, we demonstrate that the model outperforms the commonly used "Majority Vote" heuristic for inferring image labels, and is robust to both noisy and adversarial labelers. <s> BIB004 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Wi-Fi fingerprinting is a technique which can provide location in GPS-denied environments, relying exclusively on Wi-Fi signals. It first requires the construction of a database of “fingerprints”, i.e. signal strengths from different access points (APs) at different reference points in the desired coverage area. The location of the device is then obtained by measuring the signal strengths at its location, and comparing it with the different reference fingerprints in the database. The main disadvantage of this technique is the labour required to build and maintain the fingerprints database, which has to be rebuilt every time a significant change in the wireless environment occurs, such as installation or removal of new APs, changes in the layout of a building, etc. This paper investigates a new method to utilise user feedback as a way of monitoring changes in the wireless environment. It is based on a system of “points” given to each AP in the database. When an AP is switched off, the number of points associated with that AP will gradually reduce as the users give feedback, until it is eventually deleted from the database. If a new AP is installed, the system will detect it and update the database with new fingerprints. Our proposed system has two main advantages. First it can be used as a tool to monitor the wireless environment in a given place, detecting faulty APs or unauthorised installation of new ones. Second, it regulates the size of the database, unlike other systems where feedback is only used to insert new fingerprints in the database. <s> BIB005 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> For many supervised learning tasks it may be infeasible (or very expensive) to obtain objective and reliable labels. Instead, we can collect subjective (possibly noisy) labels from multiple experts or annotators. In practice, there is a substantial amount of disagreement among the annotators, and hence it is of great practical interest to address conventional supervised learning problems in this scenario. In this paper we describe a probabilistic approach for supervised learning when we have multiple annotators providing (possibly noisy) labels but no absolute gold standard. The proposed algorithm evaluates the different experts and also gives an estimate of the actual hidden labels. Experimental results indicate that the proposed method is superior to the commonly used majority voting baseline. <s> BIB006 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Distributing labeling tasks among hundreds or thousands of annotators is an increasingly important method for annotating large datasets. We present a method for estimating the underlying value (e.g. the class) of each image from (noisy) annotations provided by multiple annotators. Our method is based on a model of the image formation and annotation process. Each image has different characteristics that are represented in an abstract Euclidean space. Each annotator is modeled as a multidimensional entity with variables representing competence, expertise and bias. This allows the model to discover and represent groups of annotators that have different sets of skills and knowledge, as well as groups of images that differ qualitatively. We find that our model predicts ground truth labels on both synthetic and real data more accurately than state of the art methods. Experiments also show that our model, starting from a set of binary labels, may discover rich information, such as different "schools of thought" amongst the annotators, and can group together images belonging to separate categories. <s> BIB007 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Most current methods for 802.11-based indoor localization depend on surveys conducted by experts or skilled technicians. Some recent systems have incorporated surveying by users. Structuring localization systems "organically," however, introduces its own set of challenges: conveying uncertainty, determining when user input is actually required, and discounting erroneous and stale data. Through deployment of an organic location system in our nine-story building, which contains nearly 1,400 distinct spaces, we evaluate new algorithms for addressing these challenges. We describe the use of Voronoi regions for conveying uncertainty and reasoning about gaps in coverage, and a clustering method for identifying potentially erroneous user data. Our algorithms facilitate rapid coverage while maintaining positioning accuracy comparable to that achievable with survey-driven indoor deployments. <s> BIB008 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> Although active research has recently been conducted on received signal strength (RSS) fingerprint-based indoor localization, most of the current systems hardly overcome the costly and time-consuming offline training phase. In this paper, we propose an autonomous and collaborative RSS fingerprint collection and localization system. Mobile users track their position with inertial sensors and measure RSS from the surrounding access points. In this scenario, anonymous mobile users automatically collect data in daily life without purposefully surveying an entire building. The server progressively builds up a precise radio map as more users interact with their fingerprint data. The time drift error of inertial sensors is also compromised at run-time with the fingerprint-based localization, which runs with the collective fingerprints being currently built by the server. The proposed system has been implemented on a recent Android smartphone. The experiment results show that reasonable location accuracy is obtained with automatic fingerprinting in indoor environments. <s> BIB009 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> We show how the quality of decisions based on the aggregated opinions of the crowd can be conveniently studied using a sample of individual responses to a standard IQ questionnaire. We aggregated the responses to the IQ questionnaire using simple majority voting and a machine learning approach based on a probabilistic graphical model. The score for the aggregated questionnaire, Crowd IQ, serves as a quality measure of decisions based on aggregating opinions, which also allows quantifying individual and crowd performance on the same scale. ::: ::: We show that Crowd IQ grows quickly with the size of the crowd but saturates, and that for small homogeneous crowds the Crowd IQ significantly exceeds the IQ of even their most intelligent member. We investigate alternative ways of aggregating the responses and the impact of the aggregation method on the resulting Crowd IQ. We also discuss Contextual IQ, a method of quantifying the individual participant's contribution to the Crowd IQ based on the Shapley value from cooperative game theory. <s> BIB010 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> We show how machine learning and inference can be harnessed to leverage the complementary strengths of humans and computational agents to solve crowdsourcing tasks. We construct a set of Bayesian predictive models from data and describe how the models operate within an overall crowd-sourcing architecture that combines the efforts of people and machine vision on the task of classifying celestial bodies defined within a citizens' science project named Galaxy Zoo. We show how learned probabilistic models can be used to fuse human and machine contributions and to predict the behaviors of workers. We employ multiple inferences in concert to guide decisions on hiring and routing workers to tasks so as to maximize the efficiency of large-scale crowdsourcing processes based on expected utility. <s> BIB011 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> As a new distributed computing model, crowdsourcing lets people leverage the crowd's intelligence and wisdom toward solving problems. This article proposes a framework for characterizing various dimensions of quality control in crowdsourcing systems, a critical issue. The authors briefly review existing quality-control approaches, identify open issues, and look to future research directions. In the Web extra, the authors discuss both design-time and runtime approaches in more detail. <s> BIB012 </s> A Survey of Crowd Sensing Opportunistic Signals for Indoor Localization <s> Data Fusion Solutions. <s> In this paper, we address the problem of fusing untrustworthy reports provided from a crowd of observers, while simultaneously learning the trustworthiness of individuals. To achieve this, we construct a likelihood model of the users's trustworthiness by scaling the uncertainty of its multiple estimates with trustworthiness parameters. We incorporate our trust model into a fusion method that merges estimates based on the trust parameters and we provide an inference algorithm that jointly computes the fused output and the individual trustworthiness of the users based on the maximum likelihood framework. We apply our algorithm to cell tower local- isation using real-world data from the OpenSignal project and we show that it outperforms the state-of-the-art methods in both accuracy, by up to 21%, and consistency, by up to 50% of its predictions. <s> BIB013
Recently, a number of researchers proposed various methods BIB005 BIB009 BIB012 BIB006 to estimate the reliability of the reports and compute their aggregated output. In particular, many existing researches mostly in machine learning mainly focus on fusing multiple single-value observations combined with the assessment of a user's trustworthiness. Bachrach et al. BIB010 proposed Crowd IQ, which is a quality measure of decisions based on aggregating opinions and quantifies individual and crowd performances under the same scale. Their idea is to aggregate response IQ questionnaire based on simple major voting mechanism mixed with probabilistic graphical model-based machine learning approach. Kamar et al. BIB011 constructed a set of Bayesian predictive models within a crowdsourcing framework and also employ multiple inferences to guide the selection and schedule the workers so as to maximize the overall efficiency of large-scale crowdsourcing process. Welinder et al. BIB007 mainly deal with the image labelling problem. They proposed a way to estimate the underlying value (e.g., the class) of each image from (noisy) annotations provided by multiple annotators, which is based on the image formation and annotation process. In their work, common wisdom is to collect multiple labels for each sample and adopt "major vote" to decide on the correct labels. In the works mentioned above, the primary mechanism in aggregating different opinions is "major vote," which is widely used for centuries in almost everywhere in people's daily life, politics, and so forth. Whitehill et al. BIB004 also proposed a probabilistic model to simultaneously infer the label of each image. An interesting point they posed is that their model outperforms the common "major vote" mechanism in inferring the labels. Their work provided researchers later on with a hint that "major vote" might not be optimal in aggregating crowdsourced information though its simplicity makes it easy to implement. If we turn our eyesight to research in the field of mobile computing, a similar problem of multisensor fusion will arise. A vast literature has addressed how to integrate multisensor estimates into one single output, like covariance intersection  , covariance union BIB002 , and so forth. The limitation of such problems is that they typically fuse the estimates without modeling the trustworthiness of the users, or they only identify the unreliable estimates by some simple outlier detection methods, like kNN  , spatial weighted outlier detection (SOD) BIB003 , local outlier factor (LOF) BIB001 , and so forth. The underlying assumption of these methods is that the noise in the data is only introduced by uncalibrated or faulty sensors. And thus, an underlying problem is that the untrustworthy information introduced by the crowd is not taken into consideration in these methods. Park et al. BIB008 proposed the Voronoi regions for conveying uncertainty and reasoning about gaps in coverage and a clustering method for identifying potentially erroneous user data. Users are requested to input to improve either coverage or accuracy. Erroneous bind detection method is applied by clustering in signal space using linkage function. In the year 2013, Venanzi et al. introduced the idea of learning the trust of the contributors which construct a likelihood model of the users' trustworthiness by scaling the uncertainty of its multiple estimates with trustworthiness parameters BIB013 . This work gives a framework for data fusion for crowdsourcing applications.