Earlier studies have found that encoding model performance can vary widely across layers of LMs [1]}, [2]} and supervised speech models [3]}, [4]}. To test whether the same is true for SSL speech models, we measured prediction performance for encoding models built using each layer of each network shown in table:model-config. For comparison, we also tested encoding models using each layer of the supervised speech baseline and hand-constructed feature baselines.
