Self-supervised learning (SSL) has emerged as a popular and successful pre-training objective. In natural language processing, self-supervised language models (LM) encode diverse linguistic information and achieve excellent zero-shot performance on many language tasks [1]}, [2]}, [3]}. Capitalizing on these findings, neuroimaging studies have shown that representations extracted from LMs are highly effective at predicting brain activity elicited by natural language [4]}, [5]}, [6]}, [7]}, [8]}, [9]} and can help reveal how linguistic representations are organized across human cortex [10]}.
