A general technique [1]}, [2]} in the literature is to leverage the intrinsic correspondence between visual frames and audio waves by determining whether they are from the same video instance.
With recent advances of contrastive learning [3]}, [4]}, these approaches make positive (negative) video-audio pairs similar (dissimilar).
Based on this paradigm, AVTS [1]} designs a harder synchronization supervision, which defines positive visual-audio pairs only when they are from the identical clip of a video.
In addition to cross-modal discrimination, AVID [2]} considers within-modal discrimination, which simultaneously explores the relationship between visual-audio, visual-visual, and audio-audio pairs.
Recently, GDT [7]} develops a unified visual-audio contrastive framework by exploring more comprehensive data augmentations and modality relationship.
