task_categories:text-classification task_categories:question-answering task_categories:multiple-choice size_categories:10K<n<100K language:en language:zh license:cc-by-nc-sa-4.0 arxiv:2309.05557 region:us
---
license: cc-by-nc-sa-4.0
task_categories:
- text-classification
- question-answering
- multiple-choice
language:
- en
- zh
pretty_name: Netops
size_categories:
- 10K<n<100K
---
NetEval is a NetOps evaluation suite for foundation models, consisting of 5269 multi-choice questions. Please check [our paper](https://arxiv.org/abs/2309.05557) for more details about NetEval.

We hope NetEval could help developers track the progress and analyze the NetOps ability of their models.

## Citation

Please cite our paper if you use our dataset.

```
@misc{miao2023empirical,
      title={An Empirical Study of NetOps Capability of Pre-Trained Large Language Models}, 
      author={Yukai Miao and Yu Bai and Li Chen and Dan Li and Haifeng Sun and Xizheng Wang and Ziqiu Luo and Dapeng Sun and Xiuting Xu and Qi Zhang and Chao Xiang and Xinchi Li},
      year={2023},
      eprint={2309.05557},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```