Bayesian models qualify the model uncertainty by Bayesian neural networks (BNNs) [1]}, [2]}.
BNNs explicitly qualify model uncertainty by considering model parameters as distributions.
Specifically, BNNs consider probabilistic uncertainty, i.e., aleatoric uncertainty and epistemic uncertainty [3]}. Aleatoric only considers data uncertainty caused by statistical randomness. At the same time, epistemic refers to model uncertainty introduced by limited knowledge or ignorance in collected data.
Monte Carlo Dropout [4]} is a crucial technique to approximate variational Bayesian inference. It trains and evaluates a neural network with dropout [5]} before each layer.
BNNs have been explored for classification prediction or regression in CV applications. However, there has been less study in the NLP domain.
Few work [6]}, [7]}, [8]} empirically evaluate uncertainty estimation in text classification.
Other attempts adopt MC Dropout in deep active learning [9]}, [10]}, sentiment analysis [11]}, or machine translation [12]}.
