[rgb]0,0,0Notably, the visual-audio local correspondence has been partially studied in visual-audio localization [1]}, [2]} and representation learning [3]} methods.
For example, DMC [3]} treats the local features in image feature maps as a set of distinct components, and learns the visual and auditory subnets by co-clustering them.
Though the learned visual features perform well in image classification on small datasets, however, this method has not been validated in the challenging unsupervised video pretraining task.
In contrast to the clustering-based learning in DMC, in this paper we propose a novel attention-based method to explore the effectiveness of local correspondence in this challenging task, rather than learning exact localization between modalities.
