In this paper, computational aspects of the panel aggregation problem are addressed. Motivated primarily by applications of risk assessment, an algorithm is developed for aggregating large corpora of internally incoherent probability assessments. The algorithm is characterized by a provable performance guarantee, and is demonstrated to be orders of magnitude faster than existing tools when tested on several real-world data-sets. In addition, unexpected connections between research in risk assessment and wireless sensor networks are exposed, as several key ideas are illustrated to be useful in both fields.
@cite_20 consider a Bayesian approach to reconciling probability forecasts, whereby noisy" observations @math are assumed to arise from a coherent set @math . CAP can be viewed as a special-case of their model, since as discussed above, the solution to ) admits a Bayesian interpretation as the maximum-likelihood coherent forecasts given additive white noise corrupted observations @math . However, note that @cite_20 sought to eliminate incoherence from a single judge, whereas CAP was introduced to address the panel aggregation problem. Moreover, Osherson and Vardi were motivated by non-statistical interpretations of CAP and as here, addressed the computational issue of implementing CAP.
@cite_20
1599813212
We address the problem of ranking question answerers according to their credibility, characterized here by the probability that a given question answerer (user) will be awarded a best answer on a question given the answerer’s question-answering history. This probability (represented by θ ) is considered to be a hidden variable that can only be estimated statistically from specific observations associated with the user, namely the number b of best answers awarded, associated with the number n of questions answered. The more specific problem addressed is the potentially high degree of uncertainty associated with such credibility estimates when they are based on small numbers of answers. We address this problem by a kind of Bayesian smoothing. The credibility estimate will consist of a mixture of the overall population statistics and those of the specific user. The greater the number of questions asked, the greater will be the contribution of the specific user statistics relative to those of the overall population. We use the Predictive Stochastic Complexity (PSC) as an accuracy measure to evaluate several methods that can be used for the estimation. We compare our technique (Bayesian Smoothing (BS)) with maximum a priori (MAP) estimation, maximum likelihood (ML) estimation and Laplace smoothing.