In automatic speech recognition there has been a similar trend towards SSL-based approaches. Using autoregressive, contrastive, and masked prediction losses, SSL models learn powerful acoustic representations that are highly transferable across applications [1]}, [2]}, [3]}. Much like language models, SSL models of speech have also been shown to capture higher-order linguistic structure without explicit supervision [4]}, [5]}. This can be attributed to their ability to integrate information over a broader context than traditional acoustic filters. However, SSL models have not yet been combined with neuroimaging to study auditory processing in the human brain, where the best models are currently hand-constructed features [6]}, [7]}, [8]}, [9]} or supervised neural networks [10]}.
In this paper, we investigate the potential of SSL speech representations for predicting human cortical responses to natural speech.
