task_categories:text-generation task_categories:text2text-generation task_categories:conversational task_categories:text-classification task_categories:token-classification task_categories:table-question-answering task_categories:zero-shot-classification task_categories:question-answering task_categories:summarization task_categories:feature-extraction size_categories:1M<n<10M language:en license:mit region:us
---
license: mit
dataset_info:
  features:
  - name: id
    dtype: string
  - name: system_prompt
    dtype: string
  - name: question
    dtype: string
  - name: response
    dtype: string
  - name: reward
    dtype: float32
  splits:
  - name: train
    num_bytes: 3274600633.90245
    num_examples: 2409134
  - name: test
    num_bytes: 409325419.048775
    num_examples: 301142
  - name: validation
    num_bytes: 409325419.048775
    num_examples: 301142
  download_size: 2268645581
  dataset_size: 4093251472.0000005
task_categories:
- text-generation
- text2text-generation
- conversational
- text-classification
- token-classification
- table-question-answering
- zero-shot-classification
- question-answering
- summarization
- feature-extraction
language:
- en
size_categories:
- 1M<n<10M
arxiv:
- 2301.13688
- 2306.02707
---
# Dataset Card for Isotonic/OpenOrca-deduped

## Dataset Summary

This dataset is a deduplicated version of [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca)

*MinHash Deduplication with Jaccard Threshold = 0.80*

```
Original dataset size: 4233923
Number of duplicate clusters: 522077
Files in duplicate cluster: 2115143
Unique files in duplicate cluster: 892638
Filtered dataset size: 3011418
```