task_categories:text-classification size_categories:1K<n<10K language:en SFT region:us
---
dataset_info:
  features:
  - name: id
    dtype: string
  - name: question_stem
    dtype: string
  - name: choices
    sequence:
    - name: text
      dtype: string
    - name: label
      dtype: string
  - name: answerKey
    dtype: string
  splits:
  - name: train
    num_bytes: 895386
    num_examples: 4957
  - name: validation
    num_bytes: 95428
    num_examples: 500
  - name: test
    num_bytes: 91759
    num_examples: 500
  download_size: 609610
  dataset_size: 1082573
task_categories:
- text-classification
language:
- en
tags:
- SFT
size_categories:
- 1K<n<10K
---
# OpenBookQA

- Source: https://huggingface.co/datasets/openbookqa
- Num examples: 
  - 4,957 (train)
  - 500 (validation)
  - 500 (test)
- Language: English

```python
from datasets import load_dataset

load_dataset("vietgpt/piqa_en")
```

- Format for GPT-3

```python
def preprocess_gpt3(sample):
    question_stem = sample['question_stem']
    choices = sample['choices']['text']
    answerKey = sample['answerKey']

    if answerKey == 'A':
        output = f'\n<|correct|> {choices[0]}\n<|incorrect|> {choices[1]}\n<|incorrect|> {choices[2]}\n<|incorrect|> {choices[3]}'
    elif answerKey == 'B':
        output = f'\n<|correct|> {choices[1]}\n<|incorrect|> {choices[0]}\n<|incorrect|> {choices[2]}\n<|incorrect|> {choices[3]}'
    elif answerKey == 'C':
        output = f'\n<|correct|> {choices[2]}\n<|incorrect|> {choices[0]}\n<|incorrect|> {choices[1]}\n<|incorrect|> {choices[3]}'
    else:
        output = f'\n<|correct|> {choices[3]}\n<|incorrect|> {choices[0]}\n<|incorrect|> {choices[1]}\n<|incorrect|> {choices[2]}'

    return {'text': f'<|startoftext|><|context|> {question_stem} <|answer|> {output} <|endoftext|>'}

"""
<|startoftext|><|context|> The sun is responsible for <|answer|> 
<|correct|> plants sprouting, blooming and wilting
<|incorrect|> puppies learning new tricks
<|incorrect|> children growing up and getting old
<|incorrect|> flowers wilting in a vase <|endoftext|>
"""
```