
vae: Models were trained with KL-scaling of 0.001. We use two encoders and two decoders, such that the model has twice the number of parameters compared to the other models.

mc-ae: Models were trained with dropout between all trainable layers with probability \(p=0.2\) . We keep the same dropout rate during testing.

emsemble-ae: Each ensemble consists of 5 models, each initialised with a different random seed.

lae (posthoc): For experiments with linear layers we used the Laplace Redux [1]} implementation. For convolutions, we found it necessary to use our proposed hessian approximation. We use a diagonal approximation of the hessian in all experiments. After fitting the hessian, we optimise for the prior precision using the marginal likelihood [1]}. We use 100 MC sampling in all experiments.

lae (online): We use the exact diagonal in experiments with linear layers and the mixed diagonal approximation in all experiments with convolutional layers. We use a hessian memory factor of \(0.0001\)  and sample only 1 network per iteration. We found that it was not necessary to optimise for the prior precision when trained online.

