Figure REF  and REF  exemplary show one down- and up-sampling block. The ELU activation which is applied to each convolutional layer is not listed to keep it readable.
The encoding part has four down-sampling blocks, each of them consists of a 2x2 max pooling layer, followed by a 3x3 convolution layer, a batchnormalization layer, a dropout layer and again a 3x3 convolution layer followed by a batchnormalization layer. All convolution layers use zero padding. The dropout rate at the shallow encoding and decoding layers were set to 0.3, this rate increased up to 0.5 in the deeper layers. The decoding part consists of the same amount of four up-sampling blocks, each of them has a concatenate layer, a 3x3 convolution layer, a batchnormalization layer, a dropout layer, 3x3 convolution layers, a batchnormalization layer and a transpose layer with stride = (2,2) and zero padding. Contrary to the original recommendations from Ioffe and Szegedy[1]} it turned out that the batchnormalization layer works better if it is attached after the non-linear ELU activation function. To work with categorical labels a sigmoid activation was applied on the last layer of the U-Net. Several activation functions, optimizers and parameters were tested for this network, but they did not produce reasonable different results. For the final evaluation we used the ADAM optimizer with a factorised decreasing learning rate. The trainings setup and parameters are described in section REF . The network input was set to (32, 224, 224, 1). Within this study a 3D version of this U-Net was also implemented, but the 2D version beat the 3D network in all tests. Because of that, the following sections will only describe the results for the 2D model. The U-Net implementation from Isensee et al.[2]} was also applied to the GCN dataset. The model architecture of the original paper needed to be modified to work with the non-isotropic spatial resolution of the CMR images. Unfortunately the segmentation performance of this modified Isensee-model was outperformed by the own 2D model, which could be related to the modified layers and the case that the 3D volumes of GCN dataset are not well aligned in the axial direction.
<FIGURE>