Artificial neural networks (ANN) and deep learning (DL) have opened many new possibilities in the analysis of RS images. These methods have, among other things, increased the ability to perform accurate regression between different image modalities acquired from different sensors at possibly different times. Assuming that the multimodal images cover the same scene but their pixel measurements represent different domains, transfer learning (TL), domain adaptation (DA) [1]}, [2]}, [3]} and image translation [4]} are some theoretical frameworks of recent popularity that can be used to handle the aforementioned challenges. In this paper, we adopt the definition of image translation proposed in [4]}, which states that given sufficient training data, image-to-image translation is defined as the problem of translating one possible representation of a scene into another.
Cross-modal image translation based on generative adversarial networks (GANs) has drawn considerable attention since the architecture was proposed in 2014 [6]}. Image translation is achieved through a generative model, referred to as the generator \({G}\) , that is trained to capture the data distribution of the target domain. Simultaneously, a discriminative model, referred to as the discriminator \({D}\) , is trained to distinguish between image samples generated by \({G}\)  and images from the actual target domain. The GAN components \({G}\)  and \({D}\)  are trained in a minimax optimisation procedure, where they are adapted alternatingly while seeking to optimise conflicting performance criteria. The convergence of both benefits from the battle with the adversary as long as the alternating adaption is appropriately balanced. After training, \({G}\)  can be utilised separately to generate training data from the specific distribution. In the standard GAN setting, the generative model learns a mapping from a random noise vector \(\beta \)  to a target output image \(\widehat{z}_y\) ,In this work, \(\widehat{z}_y\)  represent ALS-based AGB predictions, see Sec. REF . while the discriminative model is trained to distinguish between the generated output image \(\widetilde{z}_y\)In this section \(\widetilde{z}_y\)  represent an approximation of \(\widehat{z}_y\) , generated from the \({G}\)  of a GAN. and the corresponding target output image \(\widehat{z}_y\)  [6]}. The whole process is illustrated in the upper part of Fig. REF . This idea was later extended to the cGAN architecture [8]}, where the learned mapping to the target output image \(\widehat{z}_y\)  is conditioned on the distribution of an input image x [4]}. In the cGAN setting the discriminative model instead learns to distinguish between a real pair or false pair of images, i.e. \(\lbrace x, \widehat{z}_y\rbrace \)  or \(\lbrace x, \widetilde{z}_x\rbrace \)In this work, \(\widetilde{z}_x\)  represent ALS-based AGB predictions generated from the \({G}\)  of a cGAN by conditioning on SAR data. See Sec. REF .. The training process of a cGAN is shown in the lower part of Fig. REF . Based on the aforementioned discussion, the definition of the research problem in this article is described as follows.
