task_categories:text-generation task_ids:language-modeling task_ids:masked-language-modeling annotations_creators:no-annotation multilinguality:monolingual size_categories:1M<n<10M source_datasets:original language:en license:cc0-1.0 doi:10.57967/hf/0800 region:us
---
annotations_creators:
- no-annotation
language:
- en
license:
- cc0-1.0
multilinguality:
- monolingual
pretty_name: arxiv-cs-ml-instruct-tune-50k
size_categories:
- 1M<n<10M
source_datasets:
- original
task_categories:
- text-generation
task_ids:
- language-modeling
- masked-language-modeling
paperswithcode_id: arxiv-cs-ml-instruct-tune-50k
---

# Dataset Card for "arxiv-cs-ml-instruct-tune-50k"

### Dataset Summary

The "ArtifactAI/arxiv-cs-ml-instruct-tune-50k" dataset consists of question-answer pairs derived from ArXiv abstracts from the following categories:
"cs.CL", "cs.AI", "cs.CV", "cs.HC", "cs.IR", "cs.RO", "cs.NE", "stat.ML". Questions are generated using the [t5-base model](https://huggingface.co/t5-base), while the answers are generated using the [GPT-3.5-turbo model](https://openai.com/chatgpt).


### Languages

English

## Dataset Structure

### Data Instances

#### train

- **Size of downloaded dataset files:** 38.4 MB

An example of 'train' looks as follows.

```
{
    "question": "What is multitask learning?", 
    "answer": "Multitask learning is a technique in machine learning that allows a single model to simultaneously learn multiple related tasks.
              Instead of training separate models for each task, multitask learning leverages shared patterns across tasks to improve overall performance.
              This can lead to faster training, better accuracy, and more efficient use of computational resources.
              For example, a model might be trained to simultaneously predict the sentiment of customer reviews and categorize them by topic.
              Both tasks require analyzing natural language, so the shared knowledge can help improve performance on both tasks."
}
```

### Data Fields

The data fields present in the dataset are as follows:

- question: a string feature representing the question.
- answer: a string feature representing the answer.

#### plain_text
- `text`: a `string` feature.

### Data Splits

 train:  50,063 question answer pairs


### Curation Rationale

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Source Data

Question-answer pairs derived from [ArXiv](https://arxiv.org/) abstracts.


#### Initial Data Collection and Normalization

The "ArtifactAI/arxiv-cs-ml-instruct-tune-50k" dataset consists of question-answer pairs derived from ArXiv abstracts. Questions are generated from ArXiv papers in the following categories:
- cs.CL
- cs.AI
- cs.CV
- cs.HC
- cs.IR
- cs.RO
- cs.NE
- stat.ML

Questions are generated using the [t5-base model](https://huggingface.co/t5-base), while the answers are generated using the [GPT-3.5-turbo model](https://openai.com/chatgpt).


### Annotations

The dataset doesn't contain annotations.

### Personal and Sensitive Information

None


#### Notice policy

Should you consider that our data contains material that is owned by you and should therefore not be reproduced here, please:

Clearly identify yourself, with detailed contact data such as an address, telephone number or email address at which you can be contacted.

Clearly identify the copyrighted work claimed to be infringed.

Clearly identify the material that is claimed to be infringing and information reasonably sufficient to allow us to locate the material.

And contact us at the following email address: matt at artifactai.com and datasets at huggingface.co

#### Take down policy

The original authors will comply to legitimate requests by removing the affected sources from the next release of the corpus.
Hugging Face will also update this repository accordingly.

### Citation Information

```
@misc{arxiv-cs-ml-instruct-tune-50k,
	title={arxiv-cs-ml-instruct-tune-50k},
	author={Matthew Kenney},
	year={2023}
}
```
