We hope this investigation helps us ask and answer more questions. It would be interesting to see if our observations about volatility and sparsity hold for other uncertainty estimation or ensemble methods like Bootstrap [1]} or Bayes by Backprop [2]}. These observations may also help us tune MCDropout-related hyperparameters, such as the best locations to place MCDropout layer–or layers capable of uncertainty estimation in general–in a model architecture.
