
We selected the FSIM [1]}, MS-SSIM [2]}, and MDSI [3]} image quality metrics to generate similarity maps because the trio collectively capture key image characteristics that are similar to the HVS [4]}, [3]}, [2]}, [1]}, [8]} as shown in Figure 2. The FSIM metric captures the luminance, contrast and structural information. For the MS-SSIM metric, we considered multiple scales of the synthesised image and its reference for contrast and structure while the MDSI map is derived by extracting the gradient and chromaticity of the pair of synthesised and reference images, respectively. We use an intensity coefficient; \( 0.3 \le \alpha \le 1\)  to specify the intensity of the maps.
The map fusion network \(M_F\)  is divided into three stages. First, we extract the feature similarity representations between the reference image \(X_{ref}\)  and the generated images \(\hat{X}_i\)  per iteration given as \( p(X_{ref}, \hat{X}_i)\)  for the aforementioned similarity metrics, where \(p(\cdot )\)  is an arbitrary function used to calculate similarity index maps; MS-SSIM, FSIM and MDSI. Secondly, the generated maps, (\(m_1, m_2, \)  and \(m_3\) ) are concatenated and pre-processed by two-layer MLP networks \(f(\cdot )\)  to form a spatial-based perceptual map representation \(M \in \mathbb {R}^{H \times W \times 1}\) .
At the last stage, the predicted future states \(I\)  is then computed as the expectation of spatial features \(M\)  and the channel-based features \( Z \in \mathbb {R}^{H \times W \times C}\) .
\(I\)  is then summed with the output of the encoder \(E\)  given as \(V=I+ E\) . The resulting output \( V \in \mathbb {R}^{H \times W \times C}\)  which is fed to decoder \(g_2\)  represents latent features that are optimized for better image quality.
<FIGURE>