TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression
The tree-based ensembles are known for their outstanding performance for
classification and regression problems characterized by feature vectors
represented by mixed-type variables from various ranges and domains. However,
considering regression problems, they are primarily designed to provide
deterministic responses or model the uncertainty of the output with a Gaussian
distribution. In this work, we introduce TreeFlow, the tree-based approach that
combines the benefits of using tree ensembles with capabilities of modeling
flexible probability distributions using normalizing flows. The main idea of
the solution is to use a tree-based model as a feature extractor and combine it
with a conditional variant of normalizing flow. Consequently, our approach is
capable of modeling complex distributions for the regression outputs. We
evaluate the proposed method on challenging regression benchmarks with varying
volume, feature characteristics, and target dimensionality. We obtain the SOTA
results on datasets with non-gaussian target distributions and competitive
results on gaussian ones compared to tree-based regression baselines.