Evaluating Aleatoric Uncertainty via Conditional Generative Models
Aleatoric uncertainty quantification seeks for distributional knowledge of
random responses, which is important for reliability analysis and robustness
improvement in machine learning applications. Previous research on aleatoric
uncertainty estimation mainly targets closed-formed conditional densities or
variances, which requires strong restrictions on the data distribution or
dimensionality. To overcome these restrictions, we study conditional generative
models for aleatoric uncertainty estimation. We introduce two metrics to
measure the discrepancy between two conditional distributions that suit these
models. Both metrics can be easily and unbiasedly computed via Monte Carlo
simulation of the conditional generative models, thus facilitating their
evaluation and training. We demonstrate numerically how our metrics provide
correct measurements of conditional distributional discrepancies and can be
used to train conditional models competitive against existing benchmarks.