task_categories:text-generation task_ids:document-retrieval language_creators:found multilinguality:multilingual size_categories:100K<n<1M source_datasets:original license:cc-by-nc-nd-4.0 code arxiv:2306.03091 region:us
RepoBench is a dataset that benchmarks repository-level code auto-completion systems.

RepoBench-C denotes RepoBench for code completion, 
which is subtask of RepoBench for next-line code prediction given both cross-file and in-file context.
@misc{liu2023repobench,
      title={RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems}, 
      author={Tianyang Liu and Canwen Xu and Julian McAuley},
      year={2023},
      eprint={2306.03091},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
---
language_creators:
- found
license:
- cc-by-nc-nd-4.0
multilinguality:
- multilingual
pretty_name: RepoBench-Completion
source_datasets:
- original
task_categories:
- text-generation
task_ids:
- document-retrieval
tags:
- code
size_categories:
- 100K<n<1M
---

# Dataset Card for RepoBench-C

## Dataset Description

- **Homepage:** https://github.com/Leolty/repobench
- **Paper:** https://arxiv.org/abs/2306.03091

## Dataset Summary

**RepoBench-C (Completion)** is a subtask of **RepoBench**([GitHub](https://github.com/Leolty/repobench), [arXiv](https://arxiv.org/abs/2306.03091)), focuing on the prediction of the next line of code, given in-file context (including several preceding lines and import statements), and cross-file context.

## Settings

- `cff`: short for cross_file_first, indicating the cross-file module in next line is first used in the current file.

- `cfr`: short for cross_file_random, indicating the cross-file module in next line is not first used in the current file.

- `if`: short for in_file, indicating the next line does not contain any cross-file module.

## Supported Tasks

- `python_cff`: python code prediction with cross-file-first setting.
- `python_cfr`: python code prediction with cross-file-random setting.
- `python_if`: python code prediction with in-file setting.
- `java_cff`: java code prediction with cross-file-first setting.
- `java_cfr`: java code prediction with cross-file-random setting.
- `java_if`: java code prediction with in-file setting.

## Loading Data

For example, if you want to load the `test` set to test your model on `Python` code prediction with `cff` setting, you can do the following: 

```python
from datasets import load_dataset

dataset = load_dataset("tianyang/repobench-c", "python_cff", split="test")
```

> Note: The `split` argument is optional. If not provided, the entire dataset will be loaded.

## Dataset Structure

```json
{
    "repo_name": "repository name of the data point",
    "file_path": "path/to/file",
    "context": "commented and concatenated cross-file context",
    "import_statement": "all import statements in the file",
    "code": "the code for next-line prediction",
    "prompt": "cross-file context + import statements + in-file code",
    "next_line": "the next line of the code"
}
```

## Licensing Information

CC BY-NC-ND 4.0

## Citation Information

```bibtex
@misc{liu2023repobench,
      title={RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems}, 
      author={Tianyang Liu and Canwen Xu and Julian McAuley},
      year={2023},
      eprint={2306.03091},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

## Contributions

Thanks to [@Leolty](https://github.com/Leolty) for adding this dataset.