As our goal is not to advance the state of the art for any particular registration task, but instead to explore the value of our loss-function in a generic setting, we use well-established 2D and 3D U-Net [1]} architectures for both registration and segmentation models, sketched in Figure REF . For the auto-encoder task, we use the same architecture, but without the shortcut connections. Each network consists of three encoder and decoder stages. Each stage consists of one batch normalization [2]}, two convolutional, and one dropout layer [3]}. After the final decoder step, we smooth the model output with three more convolutional layers. We experimented with deeper architectures but found they do not increase performance. The activation function is LeakyReLu throughout the network, Softmax for the final layer of the segmentation network, Sigmoid for the final layer of the auto-encoder, and linear for the final layer of the registration network. The stages have \(64, 128, 256\)  channels for 2d datasets, and \(32, 64, 128\)  channels for 3d.
