language:en language:zh language:ml license:apache-2.0 Instruction Cot region:us
---
language: 
  - en
  - zh
  - ml
tags:
- Instruction
- Cot
license: apache-2.0
datasets:
- dataset1
- dataset2
---

# Instruction-Finetuning Dataset Collection (Alpaca-CoT)
This repository will continuously collect various instruction tuning datasets. And we standardize different datasets into the same format, which can be directly loaded by the [code](https://github.com/PhoebusSi/alpaca-CoT) of Alpaca model.

We also have conducted empirical study on various instruction-tuning datasets based on the Alpaca model, as shown in [https://github.com/PhoebusSi/alpaca-CoT](https://github.com/PhoebusSi/alpaca-CoT).  

If you think this dataset collection is helpful to you, please `like` this dataset and `star` our [github project](https://github.com/PhoebusSi/alpaca-CoT)!

You are in a warm welcome to provide us with any non-collected instruction-tuning datasets (or their sources). We will uniformly format them, train Alpaca model with these datasets and open source the model checkpoints.

# Contribute

Welcome to join us and become a contributor to this project!
If you want to share some datasets, adjust the data in the following format:
```
example.json
[
{"instruction": instruction string,
"input": input string, # (may be empty)
"output": output string}
]
```
Folder should be like this:
```
Alpaca-CoT
|
|----example
|    |
|    |----example.json
|    |
|     ----example_context.json
...
```
Create a new pull request in [Community
](https://huggingface.co/datasets/QingyiSi/Alpaca-CoT/discussions) and publish your branch when you are ready. We will merge it as soon as we can.

# Data Usage and Resources
## Data Format
All data in this folder is formatted into the same templates, where each sample is as follows:
```
[
{"instruction": instruction string,
"input": input string, # (may be empty)
"output": output string}
]
```
## alpaca
#### alpaca_data.json
> This dataset is published by [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca). It contains 52K English instruction-following samples obtained by [Self-Instruction](https://github.com/yizhongw/self-instruct) techniques.
#### alpaca_data_cleaned.json
> This dataset is obtained [here](https://github.com/tloen/alpaca-lora). It is a revised version of `alpaca_data.json` by stripping of various tokenization artifacts. 

## alpacaGPT4
#### alpaca_gpt4_data.json
> This dataset is published by [Instruction-Tuning-with-GPT-4](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM). 
It contains 52K English instruction-following samples generated by GPT-4 using Alpaca prompts for fine-tuning LLMs.
#### alpaca_gpt4_data_zh.json
> This dataset is generated by GPT-4 using Chinese prompts translated from Alpaca by ChatGPT.

<!-- ## belle_cn
#### belle_data_cn.json
This dataset is published by [BELLE](https://github.com/LianjiaTech/BELLE). It contains 0.5M Chinese instruction-following samples, which is also generated by [Self-Instruction](https://github.com/yizhongw/self-instruct) techniques.
#### belle_data1M_cn.json
This dataset is published by [BELLE](https://github.com/LianjiaTech/BELLE). It contains 1M Chinese instruction-following samples. The data of `belle_data_cn.json` and `belle_data1M_cn.json` are not duplicated. -->

## Chain-of-Thought
#### CoT_data.json
> This dataset is obtained by formatting the combination of 9 CoT datasets published by [FLAN](https://github.com/google-research/FLAN). It contains 9 CoT tasks involving 74771 samples.
#### CoT_CN_data.json
> This dataset is obtained by tranlating `CoT_data.json` into Chinese, using Google Translate(en2cn). 
#### formatted_cot_data folder
> This folder contains the formatted English data for each CoT dataset.
#### formatted_cot_data folder
> This folder contains the formatted Chinese data for each CoT dataset.

## CodeAlpaca
#### code_alpaca.json
> This dataset is published by [codealpaca](https://github.com/sahil280114/codealpaca). It contains code generation task involving 20022 samples.

## finance
#### finance_en.json
> This dataset is collected from [here](https://huggingface.co/datasets/gbharti/finance-alpaca). It contains 68912 financial related instructions in English. 

## firefly
#### firefly.json
> his dataset is collected from [here](https://github.com/yangjianxin1/Firefly). It contains 1649398 chinese instructions in 23 nlp tasks. 

## GPT4all
#### gpt4all.json
> This dataset is collected from [here](https://github.com/nomic-ai/gpt4all). It contains 806199 en instructions in code, storys and dialogs tasks. 
#### gpt4all_without_p3.json
> gpt4all without Bigscience/P3, contains 437605 samples.

## GPTeacher
#### GPTeacher.json
> This dataset is collected from [here](https://github.com/teknium1/GPTeacher). It contains 29013 en instructions generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer. 


## Guanaco
#### GuanacoDataset.json
> This dataset is collected from [here](https://huggingface.co/datasets/JosephusCheung/GuanacoDataset). It contains 534610 en instructions generated by text-davinci-003  upon 175 tasks from the Alpaca model by providing rewrites of seed tasks in different languages and adding new tasks specifically designed for English grammar analysis, natural language understanding, cross-lingual self-awareness, and explicit content recognition.
#### Guanaco_additional_Dataset.json
> A new additional larger dataset for different languages.

## HC3
#### HC3_ChatGPT.json/HC3_Human.json
> This dataset is collected from [here](https://huggingface.co/datasets/Hello-SimpleAI/HC3). It contains 37175 en/zh instructions generated by ChatGPT and human.
#### HC3_ChatGPT_deduplication.json/HC3_Human_deduplication.json
> HC3 dataset without deduplication instructions.

## instinwild
#### instinwild_en.json & instinwild_cn.json
> The two datasets are obtained [here](https://github.com/XueFuzhao/InstructionWild). It contains 52191 English and 51504 Chinese instructions, which are collected from Twitter, where users tend to share their interesting prompts of mostly generation, open QA, and mind-storm types. (Colossal AI used these datasets to train the ColossalChat model.)

## instruct
#### instruct.json
> The two datasets are obtained [here](https://huggingface.co/datasets/swype/instruct). It contains 888969 English instructions, which are caugmentation performed using the advanced NLP tools provided by AllenAI.

## Natural Instructions
#### natural-instructions-1700tasks.zip
> This dataset is obtained [here](https://github.com/allenai/natural-instructions). It contains 5040134 instructions, which are collected from diverse nlp tasks 

## prosocial dialog
#### natural-instructions-1700tasks.zip
> This dataset is obtained [here](https://huggingface.co/datasets/allenai/prosocial-dialog). It contains 165681 English instructions, which are produuced by GPT-3 rewrites questions and humans feedback 

## xP3
#### natural-instructions-1700tasks.zip
> This dataset is obtained [here](https://huggingface.co/datasets/bigscience/xP3). It contains 78883588 instructions, which are collected by prompts & datasets across 46 of languages & 16 NLP tasks

## Chinese-instruction-collection
> all datasets of Chinese instruction collection

## combination
#### alcapa_plus_belle_data.json
> This dataset is the combination of English `alpaca_data.json` and Chinese `belle_data_cn.json`.
#### alcapa_plus_cot_data.json
> This dataset is the combination of English `alpaca_data.json` and CoT `CoT_data.json`.
#### alcapa_plus_belle_cot_data.json
> This dataset is the combination of English `alpaca_data.json`, Chinese `belle_data_cn.json` and CoT `CoT_data.json`.

## Citation
Please cite the repo if you use the data collection, code, and experimental findings in this repo. 
```
@misc{alpaca-cot,
  author = {Qingyi Si, Zheng Lin },
  school = {Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China},
  title = {Alpaca-CoT: An Instruction Fine-Tuning Platform with Instruction Data Collection and Unified Large Language Models Interface},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/PhoebusSi/alpaca-CoT}},
}
```
Cite the original Stanford Alpaca, BELLE and FLAN papers as well, please.
