
[S] VGGish and [S] VGGish-4K are from VGGish[1]} pre-trained on YouTube-8M[2]}. [S] VGGish is the output 128-d per frame embeddings used by [3]}[4]}[5]}[6]}[7]}, and [S] VGGish-4K is the 4,096-d per frame embeddings from the first FC block. We use pre-trained weights from a public repositoryhttps://github.com/tcvrick/audioset-vggish-tensorflow-to-pytorch.

[Sas] PANNs is the CNN14 from PANNs[8]} pre-trained on AudioSet. We use 2,048-d embeddings that are the input to the last FC layer, and use pre-trained weights from a public repositoryhttps://github.com/qiuqiangkong/audioset_tagging_cnn.

[Sas] ESResNe(X)t-fbsp[9]} (ESResNeXt) is a CNN pre-trained on ImageNet and AudioSet. We extract 2,048-d embeddings that are the input to the last FC layer, and use the weights from a public repositoryhttps://github.com/AndreyGuzhov/ESResNeXt-fbsp.

[Sas] AST[10]} is an Audio Spectrogram Transformer pre-trained on ImageNet and AudioSet. We use 768-d [CLS] token embeddings, and the weights from a public repositoryhttps://github.com/YuanGongND/ast.

[Ux] COALA[11]} is a cross-modal self-supervised learning of audio and tags, pre-trained on a smaller dataset consisting of 170,793 training and 19,103 validation sounds from Freesound[12]}. We use the audio encoder output, 1,152-d per frame embeddings, and pre-trained weights from a public repositoryhttps://github.com/xavierfav/coala.

[Ux] OpenL3-E and [Ux] OpenL3-M are from OpenL3[13]}, cross-modal self-supervised learning of audio and video that pre-trains the \(L^3\) -Net[14]} models on the environmental or music subset of AudioSet. We use 6,144-d per frame embeddings outputs from \(L^3\) -Nets audio encoders and pre-trained weights from a public repositoryhttps://github.com/torchopenl3/torchopenl3.

[U] TRILL[15]} is a speech representation model pre-trained on speech containing clips from AudioSet. We use layer19 12,288-d embeddings and pre-trained weights on the TensorFlow Hubhttps://tfhub.dev/google/nonsemantic-speech-benchmark/trill/2.

[U] Wav2Vec2-F and [U] Wav2Vec2-C are from Wav2Vec2.0[16]}. [U] Wav2Vec2-F is the 512-d embeddings from the front-end CNN feature encoder, and [U] Wav2Vec2-C is the 1,024-d embeddings from the Transformer context network output. We use the Wav2Vec2-Large-960h-Lv60 pre-trained weights on the Hugging Face model hubhttps://huggingface.co/facebook/wav2vec2-large-960h-lv60 pre-trained on Librispeech[17]}. Since the Wav2Vec2.0 is pre-trained on the ASR corpus only, downstream tasks other than speech context are challenging.

