A Kernelised Stein Statistic for Assessing Implicit Generative Models
Synthetic data generation has become a key ingredient for training machine
learning procedures, addressing tasks such as data augmentation, analysing
privacy-sensitive data, or visualising representative samples. Assessing the
quality of such synthetic data generators hence has to be addressed. As (deep)
generative models for synthetic data often do not admit explicit probability
distributions, classical statistical procedures for assessing model
goodness-of-fit may not be applicable. In this paper, we propose a principled
procedure to assess the quality of a synthetic data generator. The procedure is
a kernelised Stein discrepancy (KSD)-type test which is based on a
non-parametric Stein operator for the synthetic data generator of interest.
This operator is estimated from samples which are obtained from the synthetic
data generator and hence can be applied even when the model is only implicit.
In contrast to classical testing, the sample size from the synthetic data
generator can be as large as desired, while the size of the observed data,
which the generator aims to emulate is fixed. Experimental results on synthetic
distributions and trained generative models on synthetic and real datasets
illustrate that the method shows improved power performance compared to
existing approaches.