Pretraining Objectives. To evaluate the importance of MEP and CPP tasks, we test two settings: (i) Mid-w/o-CPP and (ii) Mid-w/o-MEP. On Kinetics-Sounds, these achieve 65.9% and 64.6%, respectively; ours achieve 67.5% (top-1 accuracy). The result show that the MEP task plays an important role during pretraining, confirming the findings from [1]} that the InfoNCE loss, as deployed in CBT, is effective in the cross-modal setting. The result also shows that augmenting MEP with CPP provides further performance improvement by learning cross-modal correspondence.
