task_categories:text2text-generation task_categories:conversational task_categories:text-generation task_categories:text-classification task_categories:token-classification task_categories:zero-shot-classification size_categories:1M<n<10M language:en license:apache-2.0 instructions instruction-tuning instruction-finetuning flan promptsource tasksource arxiv:2301.05948 region:us
---
dataset_info:
  features:
  - name: inputs
    dtype: string
  - name: targets
    dtype: string
  - name: task
    dtype: string
  splits:
  - name: train
    num_bytes: 2839591299.0
    num_examples: 4894553
  - name: test
    num_bytes: 97972920.0
    num_examples: 151829
  - name: validation
    num_bytes: 96766748.0
    num_examples: 148634
  download_size: 1631162334
  dataset_size: 3034330967.0
license: apache-2.0
task_categories:
- text2text-generation
- conversational
- text-generation
- text-classification
- token-classification
- zero-shot-classification
language:
- en
tags:
- instructions
- instruction-tuning
- instruction-finetuning
- flan
- promptsource
- tasksource
pretty_name: tasksource-instruct
size_categories:
- 1M<n<10M
---
# Dataset Card for "tasksource-instruct-v0" (TSI)


Multi-task instruction-tuning data recasted from 485 of the [tasksource](https://github.com/sileod/tasksource) datasets.
Dataset size is capped at 30k examples per task to foster task diversity.

```python
!pip install tasksource, pandit
import tasksource, pandit
df = tasksource.list_tasks(instruct=True).sieve(id=lambda x: 'mmlu' not in x)
for tasks in df.id:
  yield tasksource.load_task(task,instruct=True,max_rows=30_000,max_rows_eval=200)
```

https://github.com/sileod/tasksource

## How it differs from flan-v2

TSI is HuggingFace-centric and based on tasksource, a curated collection of HF datasets. It can be scaled to much more examples.
tasksource is focused on discriminative tasks (Classification/TokenClassification/MultipleChoice). The coverage on discriminative tasks is greater than flan.
List of tasks [here](https://github.com/sileod/tasksource/blob/main/tasks.md). Examples of tasks not in Flan V2 include Dynasent (adversarial sentiment analysis), Dynahate (adversarial hate speech detection, discriminative babi, epistemic logic, ruletaker, veridicality, discourse relation prediction, dozens of interesting natural language inference datasets...

TSI answers are mostly short answers to multiple-choice questions, but they target a wide array of problems.
TSI is reasoning intensive, while some flan tasks are not necessarily specific (e.g. generating hypothesis based on premise for NLI).
We explicitly mention that answers should not have explanations, to prevent biasing models toward short answers when using other instruction datasets.

`flan-v2` and `tasksource-instruct` can be combined to improve the reasoning capabilities of LLM. 

## Contact and citation:
damien.sileo@inria.fr

https://arxiv.org/abs/2301.05948
```
@article{sileo2023tasksource,
  title={tasksource: Structured Dataset Preprocessing Annotations for Frictionless Extreme Multi-Task Learning and Evaluation},
  author={Sileo, Damien},
  url= {https://arxiv.org/abs/2301.05948},
  journal={arXiv preprint arXiv:2301.05948},
  year={2023}
}
```