task_categories:text-classification task_categories:zero-shot-classification task_categories:conversational size_categories:10K<n<100K language:en license:mit arxiv:2105.03748 region:us
---
license: mit
task_categories:
- text-classification
- zero-shot-classification
- conversational
language:
- en
pretty_name: uss-ratings-dataset
size_categories:
- 10K<n<100K
---

### Dataset Description

- **Homepage:** https://github.com/sunnweiwei/user-satisfaction-simulation
- **Repository:** https://github.com/sunnweiwei/user-satisfaction-simulation
- **Paper:** https://arxiv.org/pdf/2105.03748.pdf
- **View records using Datasette:** [datasette-link](https://lite.datasette.io/?parquet=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fakomma%2Fuss-ratings-dataset%2Fresolve%2Fmain%2Fuss-ratings-dataset-datasette.parquet#/data/uss-ratings-dataset-datasette)

### Dataset Summary

- Dialogs Quality Dataset
- With both turn-level and dialog-level ratings provided on a scale of 1 to 5 by human annotators. 
- Each task has been annotated by multiple annotators.
- Contains annotated dialogs from 4 different datasets (SGD, MultiWoz, ReDial, CCPE)
- Total 34358 turns from 3500 dialogs


|Dataset|Dialogs|Turns |
|-------|------:|-----:|
|SGD    | 1000  | 11833|
|MWOZ   | 1000  | 10553|
|Redial | 1000  | 6792 |
|CCPE   | 500   | 5180 |

### Column Definitions

|Column             |Type   |Example Value            |Description                                    |
|-------------------|-------|-------------------------|-----------------------------------------------|
|split              | str   | CCPE;MWOZ;SGD;Redial    | dataset name                                  |
|session_idx        | int   | 1                       | dialog identifier                             |
|turn_idx           | int   | 1                       | turn identifier within a dialog               |
|tree_idx           | int   | 1                       | tree identifier within a turn (is all 1s here)|
|system             | str   | Do you like movies      | system message                                |
|user               | str   | No I don't like         | user message                                  |
|turn_scores        | list  | [3; 2; 2]               | list of turn-level quality scores from different human annotations|
|mean_turn_rating   | float | 2.33                    | mean of turn-level annotator scores           |
|mode_turn_rating   | int   | 2                       | mode of turn-level annotator scores           |
|dialog_scores      | list  | [3; 3; 3]               | list of dialog-level quality scores from different human annotations|
|mean_dialog_rating | float | 3.00                    | mean of dialog-level annotator scores         |
|mode_dialog_rating | int   | 3                       | mode of dialog-level annotator scores         |


### Dataset Description

- **Homepage:** https://github.com/sunnweiwei/user-satisfaction-simulation
- **Repository:** https://github.com/sunnweiwei/user-satisfaction-simulation
- **Paper:** https://arxiv.org/pdf/2105.03748.pdf
- **View records using Datasette:** [datasette-link](https://lite.datasette.io/?parquet=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fakomma%2Fuss-ratings-dataset%2Fresolve%2Fmain%2Fuss-ratings-dataset-datasette.parquet#/data/uss-ratings-dataset-datasette)

