@inproceedings{he-etal-2024-decoding,
    title = "Decoding Probing: Revealing Internal Linguistic Structures in Neural Language Models Using Minimal Pairs",
    author = "He, Linyang  and
      Chen, Peili  and
      Nie, Ercong  and
      Li, Yuanning  and
      Brennan, Jonathan R.",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.402",
    pages = "4488--4497",
    abstract = "Inspired by cognitive neuroscience studies, we introduce a novel {``}decoding probing{''} method that uses minimal pairs benchmark (BLiMP) to probe internal linguistic characteristics in neural language models layer by layer. By treating the language model as the brain and its representations as {``}neural activations{''}, we decode grammaticality labels of minimal pairs from the intermediate layers{'} representations. This approach reveals: 1) Self-supervised language models capture abstract linguistic structures in intermediate layers that GloVe and RNN language models cannot learn. 2) Information about syntactic grammaticality is robustly captured through the first third layers of GPT-2 and also distributed in later layers. As sentence complexity increases, more layers are required for learning grammatical capabilities. 3) Morphological and semantics/syntax interface-related features are harder to capture than syntax. 4) For Transformer-based models, both embeddings and attentions capture grammatical features but show distinct patterns. Different attention heads exhibit similar tendencies toward various linguistic phenomena, but with varied contributions.",
}

Inspired by cognitive neuroscience studies, we introduce a novel {``}decoding probing{''} method that uses minimal pairs benchmark (BLiMP) to probe internal linguistic characteristics in neural language models layer by layer. By treating the language model as the brain and its representations as {``}neural activations{''}, we decode grammaticality labels of minimal pairs from the intermediate layers{'} representations. This approach reveals: 1) Self-supervised language models capture abstract linguistic structures in intermediate layers that GloVe and RNN language models cannot learn. 2) Information about syntactic grammaticality is robustly captured through the first third layers of GPT-2 and also distributed in later layers. As sentence complexity increases, more layers are required for learning grammatical capabilities. 3) Morphological and semantics/syntax interface-related features are harder to capture than syntax. 4) For Transformer-based models, both embeddings and attentions capture grammatical features but show distinct patterns. Different attention heads exhibit similar tendencies toward various linguistic phenomena, but with varied contributions.
He, Linyang Chen, Peili Nie, Ercong Li, Yuanning Brennan, Jonathan R.
Decoding Probing: Revealing Internal Linguistic Structures in Neural Language Models Using Minimal Pairs