size_categories:100K<n<1M language:en code region:us
---
dataset_info:
  features:
  - name: id
    dtype: string
  - name: language
    dtype: string
  - name: license
    dtype: string
  - name: instruction
    dtype: string
  - name: output
    dtype: string
  - name: input
    dtype: string
  splits:
  - name: train
    num_bytes: 813842543
    num_examples: 491119
  download_size: 390498760
  dataset_size: 813842543
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
language:
- en
tags:
- code
size_categories:
- 100K<n<1M
---

Octocode's [CommitPackFT](https://huggingface.co/datasets/bigcode/commitpackft) in Alpaca instruction format, with several randomly selected natural language preludes to the commit messages to make them better resemble a user request.

When the instruction, old code, and new code combined are small enough to fit within 4096 Llama tokens the output is usually the full contents of the file after a commit. Otherwise, the output will be a sequence of ndiff chunks with up to five lines of context each.

An example:

    ```ndiff
      from django.conf.urls.defaults import *
      from models import GeneralPost
    - from feeds import LatestPosts
    + from feeds import latest
    ```


    ```ndiff
      post_list = {
          'queryset': GeneralPost.objects.all(),
      }

      feeds = {
    -     'all': LatestPosts,
    +     'all': latest(GeneralPost, 'dzenlog-post-list'),
    ```

This is essentially the output of `difflib.ndiff` with the `?` lines removed and large spans of unchanged text removed. The idea is that this is hopefully an easier format for a large language model to learn than a typical diff, while still being generally unambiguous enough to be useful as an output. We'll see if that works out!

Language composition:
| Language | Instructions | Percent of Instructions |
| --- | --- | --- |
| YAML | 114320 | 23.28% |
| Ruby | 69413 | 14.13% |
| Markdown | 62518 | 12.73% |
| Python | 56025 | 11.41% |
| JavaScript | 52989 | 10.79% |
| JSON | 39777 | 8.1% |
| PHP | 24791 | 5.05% |
| Java | 20635 | 4.2% |
| C# | 9346 | 1.9% |
| XML | 9337 | 1.9% |
| C | 8506 | 1.73% |
| TypeScript | 5868 | 1.19% |
| C++ | 4992 | 1.02% |
| Swift | 4849 | 0.99% |
| Rust | 2996 | 0.61% |
| Haskell | 1389 | 0.28% |
| Emacs Lisp | 1015 | 0.21% |
| Common Lisp | 778 | 0.16% |
| Erlang | 480 | 0.1% |
| OCaml | 333 | 0.07% |
| Smalltalk | 284 | 0.06% |
| Ada | 265 | 0.05% |
| Scheme | 213 | 0.04% |

All credit to the original authors of the code and the team behind OctoPack.

### Licensing Information

Each sample comes from a code repository with a permissive license. The license is provided by the `license` field for each sample.

### Citation Information

```bibtex
@article{muennighoff2023octopack,
      title={OctoPack: Instruction Tuning Code Large Language Models}, 
      author={Niklas Muennighoff and Qian Liu and Armel Zebaze and Qinkai Zheng and Binyuan Hui and Terry Yue Zhuo and Swayam Singh and Xiangru Tang and Leandro von Werra and Shayne Longpre},
      journal={arXiv preprint arXiv:2308.07124},
      year={2023}
}
```