Multi-Modal Video Adaptation. Most recently, there are also a few works integrating multiple modality data for video-based UDA. Although we only use the single modality RGB features, we still discuss this multi-modal research line for a complete literature review. The very first work exploring the multi-modal nature of videos for UDA is MM-SADA [1]}, where the correspondence of multiple modalities is exploited as a self-supervised alignment in addition to adversarial alignment. A later work, spatial-temporal contrastive domain adaptation (STCDA) [2]}, utilizes a video-based contrastive alignment as the multi-modal domain metric to measure the video-level discrepancy across domains.
[3]} propose cross-modal and cross-domain contrastive losses to handle feature spaces across modalities and domains. [4]} leverage both cross-modal complementarity and cross-modal consensus to learn the most transferable features through a CIA framework. It is worth noting that the proposed TranSVAE – although only uses single modality RGB features – surprisingly achieves better UDA performance compared to the current state-of-the-art multi-modal method CIA, which highlights our superiority.
