The multimodal alignment module, learns to project both the visual and semantic representations to a common latent embedding space \(\mathcal {Z}\) .
Let \(f(\mathbf {x}):\mathcal {X}\rightarrow \mathcal {Z}\)  denote a feature extractor, which maps an image \(\mathbf {x}\)  in the visual space \(\mathcal {X}\)  to a vector \(\mathbf {z}_{v}\)  in the latent embedding space \(\mathcal {Z}\) . Furthermore, let the function \(g\)  learn a mapping from semantic space to the latent embedding space, i.e., \(g(\mathbf {n},\mathbf {a}_y):\mathcal {N}\times \mathcal {A}\rightarrow \mathcal {Z}\)  by taking a random Gaussian noise vector \(\mathbf {n}\)  concatenated with the semantic representation \(\mathbf {a}_y\)  as input and mapping it to a vector \(\mathbf {z}_{a}\)  in \(\mathcal {Z}\) . Let \(D_1: \mathcal {Z} \times \mathcal {A} \rightarrow \mathbb {R}\)  denote a conditional discriminator (conditioned on the semantic embedding \(\mathbf {a}_y\) ) . Then, the multimodal adversarial alignment of the visual and semantic embedding spaces is achieved by employing a Wasserstein GAN [1]}, as given by
\(\mathcal {L}_{D_{1}} = \hspace{1.99997pt} \mathbb {E}[D_1(\mathbf {z}_{v}, \mathbf {a}_y)] - \mathbb {E}[D_1({\mathbf {z}_{a}}, \mathbf {a}_y)] -\lambda \mathbb {E}[\left(||\nabla _{\tilde{\mathbf {z}}} D_1(\tilde{\mathbf {z}}, \mathbf {a}_y)||_2 - 1\right)^2],\) 
