The networks are trained on a single NVIDIA TITAN X (PASCAL) GPU with 12 GB memory. To fit the training data into the GPU memory, the satellite images are tiled into 21480 samples of size 256\(\times \) 256 px. A minibatch of 4 is applied in both networks. The samples are augmented not only by horizontal and vertical flipping but also tiled from the original image with a random overlap every epoch to give the model a clue about building geometries which happened to be on the patch border in previous epochs. During the training of both networks, the ADAM optimizer is used with an initial learning rate of \(\alpha =0.0002\)  and momentum parameters \({\beta }_1=0.5, {\beta }_2=0.999\) . For the gl:DSM refinement network, the generator is pre-trained for 100 epochs as a warm-up and later interpolated with the cGAN's generator. This so-called network interpolation [1]} can balance CNN's over-smoothing and GAN's over-sharpening. The initial learnable weighting parameters as described in eq:weighting are equally set to 1.
