We present a nonparametric approach to probabilistic regression called PRESTO (Probabilistic REgression with Structured Trees). Like NGBoost, it relies on gradient boosting ([1]}, [2]}) and is easy to implement and fit on new data. Specifically, models can be fit with the same ease as standard gradient boosting or random forest models using the standard fit/predict paradigm. However, unlike NGBoost, PRESTO does not make any assumptions about the form of the conditional density. Rather, it approximates the true conditional density by fitting multiple piecewise constant densities and averaging the results. Consequently, it performs well in situations where the true conditional density is multimodal or otherwise does not have a simple or consistent parametric form.
