size_categories:100K<n<1M language:en license:apache-2.0 region:us
---
dataset_info:
  features:
  - name: prompt
    dtype: string
  - name: response
    dtype: string
  - name: source
    dtype: string
  splits:
  - name: train
    num_bytes: 1774285641
    num_examples: 808812
  download_size: 990673616
  dataset_size: 1774285641
license: apache-2.0
language:
- en
size_categories:
- 100K<n<1M
---

# Dataset Card for [GPT4All-J Prompt Generations]


## Dataset Description

Dataset used to train [GPT4All-J](https://huggingface.co/nomic-ai/gpt4all-j) and [GPT4All-J-LoRA](https://huggingface.co/nomic-ai/gpt4all-j-lora)

We release several versions of datasets

- **v1.0:** The original dataset we used to finetune GPT-J on
- **v1.1-breezy**: A filtered dataset where we removed all instances of `AI language model`
- **v1.2-jazzy**: A filtered dataset where we also removed instances like `I'm sorry, I can't answer...` and `AI language model`
- **v1.3-groovy**: The v1.2 dataset with ShareGPT and Dolly added with ~8% of semantic duplicates removed from the dataset using [Atlas](https://atlas.nomic.ai/)

The dataset defaults to `main` which is `v1.0`. To download a specific version, you can pass an argument to the keyword `revision` in `load_dataset`:

```python
from datasets import load_dataset

jazzy = load_dataset("nomic-ai/gpt4all-j-prompt-generations", revision='v1.2-jazzy')
```

- **Homepage:** [gpt4all.io](https://gpt4all.io/)
- **Repository:** [gpt4all](https://github.com/nomic-ai/gpt4all)
- **Paper:** [Technical Report](https://static.nomic.ai/gpt4all/2023_GPT4All-J_Technical_Report_2.pdf)
- **Atlas Map:** [Map of Prompts](https://atlas.nomic.ai/map/gpt4all-j-prompts-curated) and [Responses](https://atlas.nomic.ai/map/gpt4all-j-response-curated)