task_categories:conversational size_categories:n<1K language:ar language:es language:en language:hi language:fr language:zh license:apache-2.0 arxiv:2212.10560 region:us
---
license: apache-2.0
dataset_info:
  features:
  - name: prompt
    dtype: string
  splits:
  - name: ar
    num_bytes: 3010
    num_examples: 32
  - name: en
    num_bytes: 2145
    num_examples: 32
  - name: es
    num_bytes: 2474
    num_examples: 32
  - name: fr
    num_bytes: 2493
    num_examples: 32
  - name: hi
    num_bytes: 5114
    num_examples: 32
  - name: zh
    num_bytes: 1910
    num_examples: 32
  download_size: 18710
  dataset_size: 17146
task_categories:
- conversational
language:
- ar
- es
- en
- hi
- fr
- zh
size_categories:
- n<1K
---
# Dataset Card for xOA22 - Multilingual Prompts from OpenAssistant
### Dataset Summary
x-self-instruct-seed-32 consists of 32 prompts chosen out of the 252 prompts in the [self-instruct-seed](https://huggingface.co/datasets/HuggingFaceH4/self-instruct-seed) dataset from the [Self-Instruct](https://arxiv.org/pdf/2212.10560.pdf) paper. These 32 prompts were filtered out according to the following criteria:
- Should be natural in a chat setting
  - Therefore, we filter out any prompts with "few-shot examples", as these are all instruction prompts that we consider unnatural in a chat setting
- Should be well-written and easily understood
  - Our intention is to use the prompts as-is, without modification, in order to maintain parity with any other experiments that use this dataset
  - However, we planned to translate the prompts into multiple languages, and poorly written or confusing prompts could lead to high variance in the resulting translations
- Avoid asking for code / domain specific languages
  - Responses in code or domain specific languages defeat the purpose of multilingual evaluation
- Avoid potentially simple numerical responses
  - These responses would likely be the same in every language and aren't good measures of multilingual ability
- Avoid requests for translation
  - A good response will always be in the same language, so these prompts defeat the purpose of translating prompts into multiple languages
- Avoid prompts that may be difficult to translate / use English-specific language constructs
  - Prompts that rely on English constructs such as puns, dad jokes, or witty proverbs may not translate well to other languages
  - Some concepts or pop culture references may be culture-specific and difficult to translate to other languages, e.g. knowledge about American celebrities
- Avoid duplicate prompts / prompts that are too similar

The prompts were then manually translated by volunteers into 5 languages: Arabic, Simplified Chinese, French, Hindi and Spanish. 

This dataset was  originally curated for use in human evaluations of the multilingual abilities of [BLOOMChat](https://huggingface.co/sambanovasystems/BLOOMChat-176B-v1). Since not all prompts could be directly translatable due to cultural and linguistic differences, volunteers were encouraged to make appropriate substitutions and modifications that would maintain the intent of the original English prompt. We make note of any major departures from the original English prompts below.
### Languages
- Arabic (ar)
- English (en)
- Spanish (es)
- French (fr)
- Hindi (hi)
- Chinese (zh)

## Dataset Structure
### Data Fields
- `prompt`: manually translated prompt text. The English split is un-modified from the OpenAssistant Converstaions paper.

### Data Splits
The x-self-instruct-seed-32 dataset has 6 splits, one for each language. Below are the statistics for each split

| Dataset Split | Number of Instances in Split |
| ------------- | ---------------------------- |
| ar            | 32                           |
| en            | 32                           |
| es            | 32                           |
| fr            | 32                           |
| hi            | 32                           |
| zh            | 32                           |

### Translation Notes
Below are notes from volunteer translators.
- Arabic
  - No translation notes
- Spanish
  - No translation notes
- French
  - Prompt 20: Not sure how to directly translate “come up with” in this context. Using “provide” instead.
- Hindi
  - Prompt 12: Changed Seattle city to a famous destination in India called Manali
- Chinese
  - Prompt 20: Drop funny answer to make it more natural in Chinese
  - Prompt 24: Rephrase a little bit to be “use humor to overcome embarrassment” to make it more natural in Chinese

### Curation Rationale
These prompts were originally curated in order to test the multilingual abilities of the BLOOMChat model. The model's responses to these translated prompts were rated on their quality in a chat setting. Therefore, emphasis was placed on making translations as natural and understandable as possible to native speakers, and we accepted feedback and modifications to the prompts from our volunteers.

### Dataset Curators
TBA

### Contributions
TBA

### Source Data
https://huggingface.co/datasets/HuggingFaceH4/self-instruct-seed