The previous analyses explained much of why the SSL features are so capable at predicting brain data: the single best layer encompasses spectrotemporal, articulatory, and some semantic information. To get a finer-grained understanding of how linguistic representations evolve across these models, we turned away from the fMRI data and instead compared each layer directly to known linguistic features.
Inspired by prior work in computer vision [1]}, natural language processing [2]}, [3]}
and speech [4]}, [5]}, we did this by linearly probing each layer's representations for spectral features (FBANK), spectrotemporal features, phoneme identity, and word identity.
