If nothing else is stated, the models were trained with the following configuration: we used Adam optimizer [1]} with learning rate 0.001 and default Pytorch settings [2]}. The learning rate was adjusted using ReduceLROnPlateau learning rate scheduler with parameters factor=0.5 and patience=5, meaning that the learning rate was halved whenever 5 epochs had passed where the validation loss had not decreased. The mean squared error loss was used as the reconstruction loss in all models. Models where trained until convergence, defined as whenever the validation loss had not decreased for 8 epochs. Models trained on mnist and fashionmnist used a batch size of 64 and on celeba a batch size of 128 was used.
