Empirical representations of probability distributions via kernel mean embeddings
How to represent probability distributions is a fundamental issue in statistical methods, as this determines all the subsequent estimation procedures. In this thesis, we focus on the approach using positive definite kernels and reproducing kernel Hilbert spaces (RKHS) namely the framework of kernel mean embeddings. In this framework, any probability distribution is represented as an element in an RKHS, which is called the kernel mean. Since each kernel mean contains all the information about the embedded distribution, statistical inference can be conducted by estimating the kernel means of distributions which one is interested in. In general, a finite sample estimate of a kernel mean is given as a weighted sum of feature vectors. Therefore an empirical kernel mean is expressed by a weighted sample, and thus it is similar to an empirical distribution. In this thesis, we investigate this similarity between empirical kernel means and empirical distributions, and show that empirical kernel means can be treated as empirical distributions; this enables us to incorporate Monte Carlo methods into the framework of kernel mean embeddings. We first prove that a sampling method can be applied to empirical kernel means, as is commonly done to empirical distributions in the field of Monte Carlo methods. Based on this theoretical result, we develop a novel method for filtering in a state space model, which effectively combines the sampling method and a nonparametric learning approach of kernel mean embeddings. We also prove that empirical kernel means can be used for estimating expectations of functions of a broad class, similarly to empirical distributions.