CMAC aims to learn a visual encoder \(f_{v}(\cdot )\)  and an audio encoder \(f_{a}(\cdot )\)  in unsupervised manner.
R(2+1)D-18 [1]} is used as \(f_{v}(\cdot )\) , and ResNet-9 [2]} is used as \(f_{a}(\cdot )\) .
For each video instance in a batch, we randomly sample 1-second clip and produce visual modality data and audio modality data.
This makes our positive visual-audio pairs always synchronous from the same video and negative visual-audio pairs from different videos, which is different from [3]}, [4]}.
The augmentation for the visual modality contains random cropping \(128\times 128\)  and random horizontal flipping.
The augmentation for audio modality employs log mel filtering with 257 bank filters, time warping, random frequency masking, and random time masking in SpecAugment [5]}.
The contrastive learning in CMAC adopts MoCo [6]} with default settings, e.g., \(\tau =0.07\) , memory bank size as 15000, and momentum update parameter as 0.999.
The channels of cross-modal filters and contrastive head are 512 and 256, respectively.
For optimization, we use SGD optimizer with \(lr=0.01\) , weight decay \(1e-5\) , momentum \(0.9\) , and mini-batch size 128 on 8 V100 GPUs.
A gradual warp-up schedule is utilized for the first 10 epochs.
