task_categories:text-generation task_categories:conversational task_categories:text2text-generation size_categories:100K<n<1M language:en license:cc-by-3.0 region:us
---
dataset_info:
  features:
  - name: alpaca_prompt
    dtype: string
  - name: response
    dtype: string
  - name: instruction
    dtype: string
  - name: source
    dtype: string
  - name: task_name
    dtype: string
  - name: template_type
    dtype: string
  splits:
  - name: train
    num_bytes: 125656035
    num_examples: 142622
  download_size: 57912402
  dataset_size: 125656035
license: cc-by-3.0
task_categories:
- text-generation
- conversational
- text2text-generation
language:
- en
pretty_name: T
size_categories:
- 100K<n<1M
---
# Dataset Card for "open-instruct"


This dataset is a combination of:
  1. Filtered subset of [OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1)
  2. train split of [Mosaic-dolly-hhrlhf](https://huggingface.co/datasets/mosaicml/dolly_hhrlhf) (consists of [Databrick's dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) dataset and a filtered subset of [Anthropic's HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf)).
  3. Filtered subset of [conceptofmind/cot_submix_original](https://huggingface.co/datasets/conceptofmind/cot_submix_original)

## Dataset
The dataset consists of 6 columns:
1. instruction: The natural language instruction without any prompt templates (we extracted them out of the alpaca-format in Mosaic-dolly-hhrlhf)
2. alpaca_prompt: Alpaca prompt template versions of instruction
3. response: The response to the instruction
4. source: Dataset source
5. task_name
6. template_type: flan template used (zeroshot or fewshot)

## License
- It is usable for commercial purposes so long as you follow the terms of the license.


### Dataset subset licenses:

- Open-instruct-v1-dolly-hhrlhf-oasst1 (Mosaic/Dolly-HHRLHF + filtered  OASST1) - cc by 3.0 

Subset of COT SUBMIX (FROM FLAN V2) Zeroshot examples: 

- ESNLI -  MIT 
- ECQA - CDLA 1.0 - Sharing
- Strategy - MIT
- CREAK - MIT
- gsmk8 - MIT
- aqua - MIT
- qasc - Apache 2.0

Certain categories of material in the dataset include materials from the following sources, licensed under the CC BY-SA 3.0 license:

Wikipedia (various pages) - https://www.wikipedia.org/
- Copyright © Wikipedia editors and contributors.

Databricks (https://www.databricks.com)
- Copyright © Databricks

Mosaic ML (https://www.mosaicml.com/)
- Copyright © Mosaic ML

VMware
- Copyright © VMware


[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)