The Rao-Blackwell theorem is utilized to analyze and improve the scalability of inference in large probabilistic models that exhibit symmetries. A novel marginal density estimator is introduced and shown both analytically and empirically to outperform standard estimators by several orders of magnitude. The developed theory and algorithms apply to a broad class of probabilistic models including statistical relational models considered not susceptible to lifted probabilistic inference.
Rao-Blackwellized (RB) estimators have been used for inference in Bayesian networks @cite_32 @cite_28 and latent Dirichlet allocation @cite_33 with application in robotics @cite_31 and activity recognition @cite_19 . The RB theorem and estimator are important concepts in statistics @cite_3 @cite_9 .
@cite_33 @cite_28 @cite_9 @cite_32 @cite_3 @cite_19 @cite_31
2121725338 2071988465 2027226665 1484010841
The primary contribution of this paper is an algorithm capable of identifying parameters in certain mixed linear nonlinear state-space models, containing conditionally linear Gaussian substructures. More specifically, we employ the standard maximum likelihood framework and derive an expectation maximization type algorithm. This involves a nonlinear smoothing problem for the state variables, which for the conditionally linear Gaussian system can be efficiently solved using a so called Rao-Blackwellized particle smoother (RBPS). As a secondary contribution of this paper we extend an existing RBPS to be able to handle the fully interconnected model under study. SUMMARY This paper proposes a post-simulation improvement for two common Monte Carlo methods, the Accept-Reject and Metropolis algorithms. The improvement is based on a Rao-Blackwellisation method that integrates over the uniform random variables involved in the algorithms, and thus post-processes the standard estimators. We show how the Rao-Blackwellised versions of these algorithms can be implemented and, through examples, illustrate the improvement in variance brought by these new procedures. We also compare the improved version of the Metropolis algorithm with ordinary and Rao-Blackwellised importance sampling procedures for independent and general Metropolis set-ups. We analyze the following model: One person, called «helper» observes an outcome x n =(x 1 ,...,x n )∈#7B-X n of the sequence X n =(X 1 ,...,X n ) of i.i.d. RV's and the statistician gets a sample y n =(y 1 ,...,y n ) of the sequence Y n (θ,x n ) of RV's with a density Π t=1 n f(y t '(θ,x t ). The helper can give some (side) information about x n to the statistician via an encoding function s n : #7B-X n →N with rate (s n )=(1 n) log#range (s n )≤R. Based on the knowledge of s n (#7B-X n ) and y n the statistician tries to estimate θ by an estimator θ n . For a maximal mean square error we establish a Cramer-Rao type bound and, in case of a finite #7B-X, prove asymptotic achievability of this bound under certain conditions This paper introduces a new method, called the robust Bayesian estimator (RBE), to learn conditional probability distributions from incomplete data sets. The intuition behind the RBE is that, when no information about the pattern of missing data is available, an incomplete database constrains the set of all possible estimates and this paper provides a characterization of these constraints. An experimental comparison with two popular methods to estimate conditional probability distributions from incomplete data—Gibbs sampling and the EM algorithm—shows a gain in robustness. An application of the RBE to quantify a naive Bayesian classifier from an incomplete data set illustrates its practical relevance.