task_categories:conversational task_categories:question-answering size_categories:n<1K source_datasets:original language:ja license:mit region:us
---
license: mit
language:
- ja
pretty_name: Rakuda - Questions for Japanese Models
task_categories:
- conversational
- question-answering
size_categories:
- n<1K
source_datasets:
- original
---

# Rakuda - Questions for Japanese models

**Repository**: [https://github.com/yuzu-ai/japanese-llm-ranking](https://github.com/yuzu-ai/japanese-llm-ranking)

This is a set of 40 questions in Japanese about Japanese-specific topics designed to evaluate the capabilities of AI Assistants in Japanese.

The questions are evenly distributed between four categories: history, society, government, and geography. 
Questions in the first three categories are open-ended, while the geography questions are more specific.

Answers to these questions can be used to rank the Japanese abilities of models, in the same way the [vicuna-eval questions](https://lmsys.org/vicuna_eval/) are frequently used to measure the usefulness of assistants.

## Usage

```python
from datasets import load_dataset

dataset = load_dataset("yuzuai/rakuda-questions")

print(dataset)
# => DatasetDict({
#       train: Dataset({
#           features: ['category', 'question_id', 'text'],
#           num_rows: 40
#       })
#    })
```
