Smooth densities and generative modeling with unsupervised random forests
Density estimation is a fundamental problem in statistics, and any attempt to
do so in high dimensions typically requires strong assumptions or complex deep
learning architectures. An important application for density estimators is
synthetic data generation, an area currently dominated by neural networks that
often demand enormous training datasets and extensive tuning. We propose a new
method based on unsupervised random forests for estimating smooth densities in
arbitrary dimensions without parametric constraints, as well as generating
realistic synthetic data. We prove the consistency of our approach and
demonstrate its advantages over existing tree-based density estimators, which
generally rely on ill-chosen split criteria and do not scale well with data
dimensionality. Experiments illustrate that our algorithm compares favorably to
state-of-the-art deep learning generative models, achieving superior
performance in a range of benchmark trials while executing about two orders of
magnitude faster on average. Our method is implemented in easy-to-use
$\texttt{R}$ and Python packages.