task_categories:text-generation size_categories:10K<n<100K language:en instruction-finetuning region:us
---
dataset_info:
  features:
  - name: output
    dtype: string
  - name: instruction
    dtype: string
  - name: input
    dtype: string
  - name: source
    dtype: string
  splits:
  - name: instructmix_15k
    num_bytes: 10498076
    num_examples: 15000
  - name: instructmix_30k
    num_bytes: 21008700
    num_examples: 30000
  - name: instructmix_50k
    num_bytes: 34872601
    num_examples: 50000
  - name: instructmix_15k_balanced
    num_bytes: 9550701
    num_examples: 15000
  - name: instructmix_30k_balanced
    num_bytes: 19149564
    num_examples: 30000
  - name: instructmix_all
    num_bytes: 59355817
    num_examples: 87039
  download_size: 94447900
  dataset_size: 154435459
language:
- en
tags:
- instruction-finetuning
pretty_name: InstructMix
task_categories:
- text-generation
size_categories:
- 10K<n<100K
---
## Dataset Card for "InstructMix"

**Description:**

InstructMix is a versatile instruction-tuning dataset available in Alpaca format. It encompasses a variety of instruction-related tasks and sources, making it well suited for finetuning instruction following Large Language Models.

#### Included Datasets:

| Dataset Name | Size           | Type                                              | Details                                 | GitHub Repo                                     |
|--------------|----------------|---------------------------------------------------|-----------------------------------------|-------------------------------------------------|
| Alpaca_GPT4  | 52,002 examples| General Instruction                              | Generated by GPT-4 using Alpaca          | [GitHub Repo](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM)   |
| dolly 2.0    | 15,015 examples| Closed QA, Summarization, etc. (Wikipedia)       | Human Annotated                        | [GitHub Repo](https://github.com/databrickslabs/dolly) |
| Code Alpaca  | 20,022 examples| Code Generation, Editing, Optimization         | Generated by text-davinci-003           | [GitHub Repo](https://github.com/sahil280114/codealpaca) |

Credit for the data source: [alpaca-CoT](https://github.com/PhoebusSi/alpaca-CoT)

#### Dataset Splits:

InstructMix offers several dataset splits, each containing a mix of examples from the mentioned datasets.

1. **instructmix_15k**: 40% Alpaca_GPT4, 40% dolly 2.0, 20% Code Alpaca (15,000 randomly chosen samples according to weightage; in our experience this weightage gives better performance when training LLMs)
2. **instructmix_30k**: 40% Alpaca_GPT4, 40% dolly 2.0, 20% Code Alpaca (30,000 randomly chosen samples according to weightage; in our experience this weightage gives better performance when training LLMs)
3. **instructmix_50k**: 40% Alpaca_GPT4, 40% dolly 2.0, 20% Code Alpaca (50,000 randomly chosen samples according to weightage; in our experience this weightage gives better performance when training LLMs)
4. **instructmix_15k_balanced**: Equal distribution of samples from Alpaca_GPT4, dolly 2.0, and Code Alpaca (15,000 examples)
5. **instructmix_30k_balanced**: Equal distribution of samples from Alpaca_GPT4, dolly 2.0, and Code Alpaca (30,000 examples)
6. **instructmix_all**: All available samples from the mentioned datasets

**Models Trained on InstructMix:**
- [Xilabs/instructmix-llama-3b](https://huggingface.co/Xilabs/instructmix-llama-3b)

**Future Updates:**
The InstructMix family of datasets is a rapidly evolving one, with plans to incorporate more curated data for instruction tuning. The creators are currently developing a new InstructMix dataset that will include conversational data.