Audio-Visual Learning.
Early work in audio-visual learning focused on speech signals, improving audio-visual speech recognition than unimodal approaches [1]}, [2]}. Recent approaches leverage unlabeled videos from specific domains [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]} and often demonstrate on audio-visual source separation, localization, and co-segmentation. However, these approaches rely on short-term audio-visual correspondence and thus may not generalize to long-term video recognition that requires global context (as was suggested in [10]}), which this work focuses on.
