While an active line of work focuses on finding alternative approximations of BNNs [1]}, [2]}, in this work we point out that BNNs and MCDropout are unable to detect certain OOD samples. Our results advocate the need for efficient UE techniques that approximate GP-based methods and that MCDropout's predictions should be taken with a grain of salt—although this is the most efficient and frequently used method in practice.
We believe that our contribution will lead the community to collaborate on new benchmarks and tests of UE methods, in order to develop methods with theoretically guaranteed properties. This is necessary to develop future-proof UE technologies that can be used for safety-critical applications in the years to come.
