We pretrain our model on Kinetics-700 [1]} or AudioSet [2]} and finetune it on various downstream tasks. The official release of Kinetics-700 contains 10-second clips only, so we download 410K original videos from YouTube and take 30-second clips from each video. For fair comparison with prior work, we use 10-second clips from the official release of AudioSet (we used 1.8M clips).
We pretrain our model on 64 NVIDIA Tesla V100 GPUs with a batch size of 256 for 220K iterations. For downstream tasks, we evaluate on short-video/audio classification using UCF-101 [3]} (13K clips from 101 classes; 7.2 seconds on average)
and ESC-50 [2]} (2K clips from 50 classes; 5 seconds), and on long-video classification using Kinetics-Sounds [5]} (23K videos from 32 classes; 10 seconds on average) and Charades [6]} (10K videos from 157 classes; 30 seconds on average). We describe various details about experimental setup in Appendix. Code will be released.
