A Geometric Method for Improved Uncertainty Estimation in Real-time
Machine learning classifiers are probabilistic in nature, and thus inevitably
involve uncertainty. Predicting the probability of a specific input to be
correct is called uncertainty (or confidence) estimation and is crucial for
risk management. Post-hoc model calibrations can improve models' uncertainty
estimations without the need for retraining, and without changing the model.
Our work puts forward a geometric-based approach for uncertainty estimation.
Roughly speaking, we use the geometric distance of the current input from the
existing training inputs as a signal for estimating uncertainty and then
calibrate that signal (instead of the model's estimation) using standard
post-hoc calibration techniques. We show that our method yields better
uncertainty estimations than recently proposed approaches by extensively
evaluating multiple datasets and models. In addition, we also demonstrate the
possibility of performing our approach in near real-time applications. Our code
is available at our Github
https://github.com/NoSleepDeveloper/Geometric-Calibrator.