In mSLAM, the parameters of the speech tokenizer are learned from scratch by a contrastive loss [1]} over a speech-only encoder. For simplicity, we directly utilize the pretrained speech tokenizer in mSLAM and keep it constant during our model training.
