license:mit region:us
We introduce Curriculum as a new format of NLI benchmark for evaluation of broad-coverage linguistic phenomena. 
Curriculum contains a collection of datasets that covers 36 types of major linguistic phenomena and an evaluation procedure 
for diagnosing how well a language model captures reasoning skills for distinct types of linguistic phenomena. 
We show that this linguistic-phenomena-driven benchmark can serve as an effective tool for diagnosing 
model behavior and verifying model learning quality.
@misc{https://doi.org/10.48550/arxiv.2204.06283,
  doi = {10.48550/ARXIV.2204.06283},
  url = {https://arxiv.org/abs/2204.06283},
  author = {Chen, Zeming and Gao, Qiyue},
  keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding},
  publisher = {arXiv},
  year = {2022},
  copyright = {Creative Commons Attribution 4.0 International}
}
---
license: mit
---
