region:us

# Dataset Card for **Data Provenance Initiative - Commercially-Verified-Licenses**

## Dataset Description

- **Homepage:** https://github.com/Data-Provenance-Initiative/Data-Provenance-Collection
- **Repository:** https://github.com/Data-Provenance-Initiative/Data-Provenance-Collection
- **Paper:** https://genlaw.github.io/CameraReady/20.pdf
- **Point of Contact:** data.provenance.init@gmail.com
- **NOTE:** Licenses for these datasets are "self-reported" and collected by best-effort volunteers on a per dataset basis. Please find more details in the paper linked above.

### Legal Disclaimer / Notice

Collected License Information is **NOT** Legal Advice.
It is important to note we collect self-reported licenses, from the papers and repositories that released these datasets, and categorize them according to our best efforts, as a volunteer research and transparency initiative.
The information provided by any of our works and any outputs of the Data Provenance Initiative do not, and are not intended to, constitute legal advice; instead, all information, content, and materials are for general informational purposes only.
Readers and users should seek their own legal advice from counsel in their relevant jurisdiction.

### Dataset Summary

A wave of recent language models have been powered by large collections of natural language datasets. The sudden race to train models on these disparate collections of incorrectly, ambiguously, or under-documented datasets has left practitioners unsure of the legal and qualitative characteristics of the models they train. To remedy this crisis in data transparency and understanding, in a joint effort between experts in machine learning and the law, we’ve compiled the most detailed and reliable metadata available for data licenses, sources, and provenance, as well as fine-grained characteristics like language, text domains, topics, usage, collection time, and task compositions. Beginning with nearly 40 popular instruction (or “alignment”) tuning collections, we release a suite of open source tools for downloading, filtering, and examining this training data. Our analysis sheds light on the fractured state of data transparency, particularly with data licensing, and we hope our tools will empower more informed and responsible data-centric development of future language models.

### What does **Commercial** mean here?
- `Commercial` includes datasets that are compatible with commercial usage, meaning commercial usage of this dataset is permitted as per its license.

### Constituent Data Collections
- Following table shows each constituent data collection this Dataset along with original source from where each data collection is derived from.

|  #  |  Collection Name  |  Description  |  Source  | 
|  ---------------  |  ---------------  |  ---------------  |  ---------------  |
|  1  | Anthropic HH-RLHF  |  Human preference data about helpfulness and harmlessness & Human-generated and annotated red teaming dialogues. | https://huggingface.co/datasets/Anthropic/hh-rlhf  |
|  2  | CommitPackFT  |   CommitPackFT is a 2GB filtered version of CommitPack to contain only high-quality commit messages that resemble natural language instructions.  |  https://huggingface.co/datasets/bigcode/commitpackft  |
|  3  | Dolly 15k  |   Databricks Dolly 15k is a dataset containing 15,000 high-quality human-generated prompt / response pairs specifically designed for instruction tuning large language models.  | https://huggingface.co/datasets/databricks/databricks-dolly-15k  |
|  4  | Flan Collection (Chain-of-Thought)  |   Chain-of-Thought sub-mixture in Flan collection dataset.  | https://huggingface.co/datasets/conceptofmind/cot_submix_original  |
|  5  | Flan Collection (Dialog)  |   Chain-of-Thought sub-mixture in Flan collection dataset.  | https://huggingface.co/datasets/conceptofmind/dialog_submix_original  |
|  6  | Flan Collection (Flan 2021)  |   Flan 2021 sub-mixture in Flan collection dataset.  | https://huggingface.co/datasets/conceptofmind/flan2021_submix_original  |
|  7  | Flan Collection (P3)  |   P3 sub-mixture in Flan collection dataset.  | https://huggingface.co/datasets/conceptofmind/t0_submix_original  |
|  8  | Flan Collection (Super-NaturalInstructions)  |   Super-Natural Instructions in Flan collection dataset.  | https://huggingface.co/datasets/conceptofmind/niv2_submix_original  |
|  9  | Joke Explanation  |   Corpus for testing whether your LLM can explain the joke well.  | https://huggingface.co/datasets/theblackcat102/joke_explaination  |
|  10  | OIG  |  Open Instruction Generalist is a large instruction dataset of medium quality along with a smaller high quality instruction dataset (OIG-small-chip2).  | https://huggingface.co/datasets/laion/OIG  |
|  11  | Open Assistant  |   OpenAssistant Conversations (OASST1) is a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 fully annotated conversation trees.  | https://huggingface.co/datasets/OpenAssistant/oasst1  |
|  12  | Open Assistant OctoPack  |   Filtered version of OpenAssistant Conversations (OASST1) to focus only on high-quality conversation trees as used in OctoPack paper.  | https://huggingface.co/datasets/bigcode/oasst-octopack  |
|  13  | Self-Instruct  |   Dataset consisting of self-generated instructions by model to improve the instruction-following capabilities of language models.  | https://huggingface.co/datasets/yizhongw/self_instruct  |
|  14  | StarCoder Self-Instruct  |   Dataset generated by prompting starcoder to generate new instructions based on some human-written seed instructions.  | https://huggingface.co/datasets/codeparrot/self-instruct-starcoder  |
|  15  | Tasksource Instruct  |   Tasksource datasets as instructions for instruction-tuning.  | https://github.com/sileod/tasksource  |
|  16  | Tasksource Symbol-Tuning  |   Tasksource datasets converted for symbol-tuning.  | https://github.com/sileod/tasksource  |
|  17  | xp3x  |   xP3x is a collection of prompts & datasets across 277 languages & 16 NLP tasks. It contains all of xP3 + much more.  | https://huggingface.co/datasets/Muennighoff/xP3x  |
|  18  | Tiny Stories  |   Dataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.  | https://huggingface.co/datasets/roneneldan/TinyStories
|  19  | Open Orca |    Open Orca dataset is a rich collection of augmented FLAN data aligns, as best as possible, with the distributions outlined in the Orca paper  | https://huggingface.co/datasets/Open-Orca/OpenOrca
|  20  | Longform |   LongForm dataset is created by leveraging English corpus examples with augmented instructions.  | https://huggingface.co/datasets/akoksal/LongForm
|  21  | HC3 (Chinese) |   HC3 dataset provides human and ChatGPT responses for different domains and languages for comparison. This is for Chinese language.  | https://huggingface.co/datasets/Hello-SimpleAI/HC3-Chinese
|  22  | Unnatural Instructions |   Unnatural Instructions is a dataset of instructions automatically generated by a Large Language model.  | https://huggingface.co/datasets/mrm8488/unnatural-instructions
|  23  | OpenAI (WebGPT) |   This is the dataset of all comparisons (model vs human answer) that were marked as suitable for reward modeling by the end of the WebGPT project. | https://huggingface.co/datasets/openai/webgpt_comparisons
|  24  | HC3 (English) |   HC3 dataset provides human and ChatGPT responses for different domains and languages for comparison. This is for English language. | https://huggingface.co/datasets/Hello-SimpleAI/HC3
|  25  | Gorilla |   Gorilla is a comprehensive dataset consisting of HuggingFace, TorchHub, and TensorHub APIs.  |  https://huggingface.co/datasets/gorilla-llm/APIBench


### Data Instances

[More Information Needed]

### Data Fields
The following snippet shows the fields in a row in each data collection in this dataset:
```
[
        {"from": "user", "text": input_text.strip(), "parent": dset},
        {"from": "assistant", "text": target_text.strip(), "parent": 0},
        ...
]
```
with fields:
- from: indicates the originator of the text in this conversation. This can be either "user" or "assistant", where "assistant" indicates the model and text will be model's response to user's text.
- text: indicates text that originator wants to communicate to receiver.
- parent: field indicating the parent for tracing the conversation hierarchy.
Here each row contains one or more json objects indicating user-assistant interaction dialogue with text messages exchanged between them. You can leverager `parent` field in json object to follow the tree structure of interactions.

### Downloading Dataset

You can load the entire dataset by using the following code:
```python
import os
from datasets import load_dataset

# If the dataset is gated/private, make sure you have run huggingface-cli login
dataset = load_dataset("DataProvenanceInitiative/Commercially-Verified-Licenses")
```

You can load a specific dataset subset such as Dolly 15k using the following code:
```python
import os
from datasets import load_dataset

subset = load_dataset(
    "json",
    split="train",
    num_proc = os.cpu_count(),
    revision="main", 
    data_files="./dolly_15k/*.jsonl"
)
```

### Data Splits

[More Information Needed]
[TODO: Add each dataset and add # of samples in train/dev]

## Dataset Creation

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

```
@misc{dpi,
  title={The Data Provenance Project},
  author={Longpre, Shayne and Mahari, Robert and Muennighoff, Niklas and Chen, Anthony and Perisetla, Kartik and Brannon, William and Kabbara, Jad and Villa, Luis and Hooker, Sara},
  journal={arXiv preprint [TODO]add id},
  year={2023}
}
```

### Contributions

Thanks to [data.provenance.init@gmail.com](mailto:data.provenance.init@gmail.com) for adding this dataset.