size_categories:100K<n<1M language:en code region:us
---
dataset_info:
- config_name: adequately_rated
  features:
  - name: id
    dtype: string
  - name: rating
    struct:
    - name: analysis
      dtype: string
    - name: judge
      dtype: string
    - name: score
      dtype: int64
  - name: language
    dtype: string
  - name: license
    dtype: string
  - name: instruction
    dtype: string
  - name: output
    dtype: string
  - name: input
    dtype: string
  splits:
  - name: train
    num_bytes: 502380874.99241877
    num_examples: 231589
  download_size: 233165301
  dataset_size: 502380874.99241877
- config_name: best_rated
  features:
  - name: id
    dtype: string
  - name: rating
    struct:
    - name: analysis
      dtype: string
    - name: judge
      dtype: string
    - name: score
      dtype: int64
  - name: language
    dtype: string
  - name: license
    dtype: string
  - name: instruction
    dtype: string
  - name: output
    dtype: string
  - name: input
    dtype: string
  splits:
  - name: train
    num_bytes: 7807230.779949458
    num_examples: 3599
  download_size: 3443289
  dataset_size: 7807230.779949458
- config_name: default
  features:
  - name: id
    dtype: string
  - name: rating
    struct:
    - name: analysis
      dtype: string
    - name: judge
      dtype: string
    - name: score
      dtype: int64
  - name: language
    dtype: string
  - name: license
    dtype: string
  - name: instruction
    dtype: string
  - name: output
    dtype: string
  - name: input
    dtype: string
  splits:
  - name: train
    num_bytes: 668703742
    num_examples: 308261
  download_size: 306198304
  dataset_size: 668703742
- config_name: ratings_only
  features:
  - name: success
    dtype: bool
  - name: score
    dtype: int64
  - name: response
    dtype: string
  - name: id
    dtype: string
  splits:
  - name: train
    num_bytes: 124887856
    num_examples: 308261
  download_size: 58208563
  dataset_size: 124887856
- config_name: worst_rated
  features:
  - name: id
    dtype: string
  - name: rating
    struct:
    - name: analysis
      dtype: string
    - name: judge
      dtype: string
    - name: score
      dtype: int64
  - name: language
    dtype: string
  - name: license
    dtype: string
  - name: instruction
    dtype: string
  - name: output
    dtype: string
  - name: input
    dtype: string
  splits:
  - name: train
    num_bytes: 10393009.91018001
    num_examples: 4791
  download_size: 4676994
  dataset_size: 10393009.91018001
configs:
- config_name: adequately_rated
  data_files:
  - split: train
    path: adequately_rated/train-*
- config_name: best_rated
  data_files:
  - split: train
    path: best_rated/train-*
- config_name: default
  data_files:
  - split: train
    path: data/train-*
- config_name: ratings_only
  data_files:
  - split: train
    path: ratings_only/train-*
- config_name: worst_rated
  data_files:
  - split: train
    path: worst_rated/train-*
language:
- en
tags:
- code
size_categories:
- 100K<n<1M
---


This is [commitpack-ft-instruct](https://huggingface.co/datasets/chargoddard/commitpack-ft-instruct), derived from Octocode's [CommitPackFT](https://huggingface.co/datasets/bigcode/commitpackft), augmented with a quality analysis of the instruction-response pair by a local model. This did a pretty decent job of identifying pairs that obviously don't have enough context to know what change is being requested, or where the commit message does not match with the changes made.

Data files (yaml, plain text, json, etc.) were heavily downsampled in preparing this dataset to skew it more towards actual code work. All entries should fit in a 4096 token context window, depending on the prompt format.

Language composition for the default configuration:
| Language | Instructions | Percent of Instructions |
| --- | --- | --- |
| Ruby | 69412 | 14.13% |
| Python | 56024 | 11.41% |
| JavaScript | 52989 | 10.79% |
| PHP | 24791 | 5.05% |
| YAML | 21764 | 4.43% |
| Java | 20635 | 4.2% |
| Markdown | 11950 | 2.43% |
| C# | 9346 | 1.9% |
| C | 8506 | 1.73% |
| JSON | 7616 | 1.55% |
| TypeScript | 5868 | 1.19% |
| C++ | 4992 | 1.02% |
| Swift | 4849 | 0.99% |
| Rust | 2996 | 0.61% |
| XML | 1766 | 0.36% |
| Haskell | 1389 | 0.28% |
| Emacs Lisp | 1015 | 0.21% |
| Common Lisp | 778 | 0.16% |
| Erlang | 480 | 0.1% |
| OCaml | 333 | 0.07% |
| Smalltalk | 284 | 0.06% |
| Ada | 265 | 0.05% |
| Scheme | 213 | 0.04% |

All credit to the original authors of the code and the team behind OctoPack.

### Licensing Information

Each sample comes from a code repository with a permissive license. The license is provided by the `license` field for each sample.

### Citation Information

```bibtex
@article{muennighoff2023octopack,
      title={OctoPack: Instruction Tuning Code Large Language Models}, 
      author={Niklas Muennighoff and Qian Liu and Armel Zebaze and Qinkai Zheng and Binyuan Hui and Terry Yue Zhuo and Swayam Singh and Xiangru Tang and Leandro von Werra and Shayne Longpre},
      journal={arXiv preprint arXiv:2308.07124},
      year={2023}
}
```