We pretrain our model on Kinetics-700 [1]} with a batch size 256 for 220K iterations and AudioSet [2]} with a batch size 300 for 220K iterations in the main experiments; for the ablation study, we use a much smaller batch size of 4 and pretrain our model on Kinetics-700 for 80K iterations.
