task_categories:text-classification size_categories:1K<n<10K language:en license:apache-2.0 region:us
---
license: apache-2.0
task_categories:
- text-classification
language:
- en
size_categories:
- 1K<n<10K
pretty_name: 'DeTexD: A Benchmark Dataset for Delicate Text Detection'
dataset_info:
  features:
  - name: text
    dtype: string
  - name: annotator_1
    dtype: int32
  - name: annotator_2
    dtype: int32
  - name: annotator_3
    dtype: int32
  - name: label
    dtype:
      class_label:
        names:
          '0': negative
          '1': positive
  splits:
  - name: test
    num_examples: 1023
---
# Dataset Card for DeTexD: A Benchmark Dataset for Delicate Text Detection

## Dataset Description

- **Repository:** [DeTexD repository](https://github.com/grammarly/detexd)
- **Paper:** [DeTexD: A Benchmark Dataset for Delicate Text Detection](TODO)

### Dataset Summary

We define *delicate text* as any text that is emotionally charged or potentially triggering such that engaging with it has the potential to result in harm. This broad term covers a range of sensitive texts that vary across four major dimensions: 1) riskiness, 2) explicitness, 3) topic, and 4) target. 

This dataset contains texts with fine-grained individual annotator labels from 0 to 5 (where 0 indicates no risk and 5 indicates high risk) and averaged binary labels. See paper for more details.

**Repository:** [DeTexD repository](https://github.com/grammarly/detexd) <br>
**Paper:** [DeTexD: A Benchmark Dataset for Delicate Text Detection](TODO)

## Dataset Structure

### Data Instances

```
{'text': '"He asked me and the club if we could give him a couple of days off just to clear up his mind and he will be back in the group, I suppose, next Monday, back for training and then be a regular part of the whole squad again," Rangnick said.',
 'annotator_1': 0,
 'annotator_2': 0,
 'annotator_3': 0,
 'label': 0}
```

### Data Fields

- `text`: Text to be classified
- `annotator_1`: Annotator 1 score (0-5)
- `annotator_2`: Annotator 2 score (0-5)
- `annotator_3`: Annotator 3 score (0-5)
- `label`: Averaged binary score (>=3), either "negative" (0) or positive (1)

### Data Splits

|                    | test |
|--------------------|-----:|
| Number of examples | 1023 |

### Citation Information

```
@inproceedings{chernodub-etal-2023-detexd,
    title = "{D}e{T}ex{D}: A Benchmark Dataset for Delicate Text Detection",
    author = "Yavnyi, Serhii and Sliusarenko, Oleksii  and Razzaghi, Jade  and Mo, Yichen  and Hovakimyan, Knar and Chernodub, Artem",
    booktitle = "The 7th Workshop on Online Abuse and Harms (WOAH)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.woah-1.2",
    pages = "14--28",
    abstract = "Over the past few years, much research has been conducted to identify and regulate toxic language. However, few studies have addressed a broader range of sensitive texts that are not necessarily overtly toxic. In this paper, we introduce and define a new category of sensitive text called {``}delicate text.{''} We provide the taxonomy of delicate text and present a detailed annotation scheme. We annotate DeTexD, the first benchmark dataset for delicate text detection. The significance of the difference in the definitions is highlighted by the relative performance deltas between models trained each definitions and corpora and evaluated on the other. We make publicly available the DeTexD Benchmark dataset, annotation guidelines, and baseline model for delicate text detection.",
}
```