Uncertainty-based Learning. Deep neural networks have achieved great success in various tasks. However since most deep models are essentially deterministic functions, the uncertainty of the model cannot be obtained. Bayesian neural networks (BNNs) [1]}, [2]}, [3]} endow deep models with uncertainty by replacing the deterministic weight parameters with distributions. Since BNNs struggle in performing inference and usually come with prohibitive computational costs, a more scalable and practical approach, MC-dropout [4]}, was proposed. In this model, the inference is completed by performing dropout sampling from the weight during training and testing. Ensemble based methods [5]} train and integrate multiple deep networks and also achieve promising performance.
Instead of indirectly modeling uncertainty through network weights, the algorithm [6]} introduces the subjective logic theory to directly model uncertainty without ensemble or Monte Carlo sampling. Building upon RBF networks, the distance between test samples and prototypes can be used as the agency for deterministic uncertainty [7]}. Benefiting from the learned weights of different tasks with homoscedastic uncertainty learning, [8]} achieves impressive performance in multi-task learning.
