task_categories:text-generation task_categories:conversational task_categories:text2text-generation size_categories:100K<n<1M language:ru license:mit region:us
---
license: mit
task_categories:
- text-generation
- conversational
- text2text-generation
language:
- ru
size_categories:
- 100K<n<1M
pretty_name: ref-chat
---

---

This dataset is based on [SiberianPersonaChat Dataset](https://huggingface.co/datasets/SiberiaSoft/SiberianPersonaChat).

It was additionally filtered using:
- politics filter ([cointegrated/rubert-base-cased-nli-threeway](https://huggingface.co/cointegrated/rubert-base-cased-nli-threeway))
- toxicity filter ([cointegrated/rubert-tiny-toxicity](https://huggingface.co/cointegrated/rubert-tiny-toxicity))
- low quality qa pairs filter ([Andrilko/ruBert-base-reward](https://huggingface.co/Andrilko/ruBert-base-reward))

**Dataset Statistics:**

- wiki_qa: 4.746
- dialog_personal_context: 68.296
- russianinstructions2: 4.812
- yandexQ_instruct: 6.316
- rugpt4: 5.269
- trupalpaca: 4.284
- text_qa: 2.57
- long_answers_qa: 3.363
- chitchat: 0.198
- reaction: 0.108
- baby: 0.037

---
### Citation

```
@MISC{morzecrew/RefinedPersonaChat,
    author  = {Yuri Zaretskiy, Nikolas Ivanov, Igor Kuzmin},
    title   = {Refined dataset for conversational agents},
    url     = {https://huggingface.co/datasets/morzecrew/RefinedPersonaChat},
    year    = 2023
}
```