size_categories:1K<n<10K language:en license:apache-2.0 dialogue segmentation region:us
\
\
---
license: apache-2.0
language:
- en
tags:
- dialogue segmentation
size_categories:
- 1K<n<10K
---

# Dataset Card for SuperDialseg

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:**
- **Repository:** 
- **Paper:** SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation
- **Leaderboard:** [https://github.com/Coldog2333/SuperDialseg](https://github.com/Coldog2333/SuperDialseg)
- **Point of Contact:** jiangjf@is.s.u-tokyo.ac.jp

### Dataset Summary

[More Information Needed]

### Supported Tasks and Leaderboards

[More Information Needed]

### Languages: English

## Dataset Structure

### Data Instances

```
{
  "dial_data": {
    "super_dialseg": [
      {
        "dial_id": "8df07b7a98990db27c395cb1f68a962e",
        "turns": [
          {
            "da": "query_condition",
            "role": "user",
            "turn_id": 0,
            "utterance": "Hello, I forgot o update my address, can you help me with that?",
            "topic_id": 0,
            "segmentation_label": 0
          },
          ...
          {
            "da": "respond_solution",
            "role": "agent",
            "turn_id": 11,
            "utterance": "DO NOT contact the New York State DMV to dispute whether you violated a toll regulation or failed to pay the toll , fees or other charges",
            "topic_id": 4,
            "segmentation_label": 0
          }
        ],
      ...
      }
    ]
}
```

### Data Fields

#### Dialogue-Level
+ `dial_id`: ID of a dialogue;
+ `turns`: All utterances of a dialogue.
#### Utterance-Level
+ `da`: Dialogue Act annotation derived from the original DGDS dataset;
+ `role`: Role annotation derived from the original DGDS dataset;
+ `turn_id`: ID of an utterance;
+ `utterance`: Text of the utterance;
+ `topic_id`: ID (order) of the current topic;
+ `segmentation_label`: 1: it is the end of a topic; 0: others.

### Data Splits

SuperDialseg follows the dataset splits of the original DGDS dataset.

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

SuperDialseg was built on the top of doc2dial and MultiDoc2dial datasets.
Please refer to the original papers for more details.

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

The annotation of dialogue segmentation points is constructed by a set of well-designed strategy. Please refer to the paper for more details.

Other annotations like Dialogue Act and Role information are derived from doc2dial and MultiDoc2dial datasets.

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

Apache License Version 2.0, following the licenses of doc2dial and MultiDoc2dial.

### Citation Information

Coming soon

### Contributions

Thanks to [@Coldog2333](https://github.com/Coldog2333) for adding this dataset.
