
As discussed above, the multimodal alignment and conceptual partitioning result in a structured and domain-agnostic latent embedding space that disentangles semantic and domain-specific information.
Such a disentanglement of semantic and domain-specific information is sufficient for standard domain-generalization setting where images during training and testing come from same categories. However in our ZSLDG setting, the
disentanglement may not hold for unseen semantic categories during testing, as previously found in [1]}.
In order to address this issue and enable generalization to unseen classes in unseen domains, we propose to learn the domain-invariance w.r.t. the joint distribution of visual and semantic representations of a class.
Formally, any given image \(\mathbf {x}\)  comprises of a class-specific content \(\textbf {C}\)  and a domain-specific transformation \(T(\cdot )\)  which depicts the class in that particular domain \(d\) . Thus, each image \(\mathbf {x} \in \mathcal {X}\)  belonging to domain \(d_{i}\)  can be represented as \(\mathbf {x} = T_{i}(\textbf {C})\) .
In order to enable generalization to unseen class in unseen domains, we propose to match the visual-semantic joint distribution \(p(T(\textbf {C}),\mathbf {a}_y)\)  under different transformations \(T_{i}(\cdot )\)  (or domains). Since the semantic space is shared between seen and unseen classes, learning domain invariance w.r.t. the joint distribution of visual and semantic representations of a class, i.e., \(p(f(T(\textbf {C})),\mathbf {a}_y)\)  or \(p(f(\mathbf {x}),\mathbf {a}_y)\)  enables us to enhance generalization.
