task_categories:sentence-similarity language:en license:cc text style transfer arxiv:2206.09676 arxiv:1909.05855 region:us
---
language:
- en
pretty_name: Schema-Guided Dialogue Dataset for Text Style Transfer
tags:
- text style transfer
license: cc
task_categories:
- sentence-similarity
viewer: true
---


# Overview

SGDD-TST - [Schema-Guided Dialogue Dataset for Text Style Transfer](https://arxiv.org/abs/2206.09676) is a dataset for evaluating the quality of content similarity measures for text style transfer in the domain of the personal plans. The original texts were obtained from [The Schema-Guided
Dialogue Dataset](https://arxiv.org/pdf/1909.05855.pdf) and were paraphrased by the [T5-based model](https://huggingface.co/ceshine/t5-paraphrase-paws-msrp-opinosis) trained on [GYAFC formality dataset](https://aclanthology.org/N18-1012/). The results were annotated by the crowdsource workers using [Yandex.Toloka](https://toloka.yandex.ru/).


# File description

The file consists of the following columns
- INPUT:text_first - the original text
- INPUT:text_second - formality transferred text
- OUTPUT:result - automatically assigned the label of the annotation (David-Skene aggregation method is used)
- CONFIDENCE:result - confidence of the annotation
- vote_type - 
- vote_different - number of votes for the option "The texts are completely different"
- vote_some_details_lost  - number of votes for the option "The texts are similar but have significant differences"
- vote_OK - number of votes for the option "The texts mean the same or have minor differences"
- **average - an averaged score of content similarity. This score can be used for evaluating the quality of content similarity measures, e.g. by calculating the Spearman Rank Correlation Coefficient between these scores and automatic scores**



# Contact and Citations

If you have any questions feel free to drop a line to [Nikolay](mailto:bbkhse@gmail.com)

If you find this repository helpful, feel free to cite our publication:

```
@InProceedings{10.1007/978-3-031-08473-7_40,
author="Babakov, Nikolay
and Dale, David
and Logacheva, Varvara
and Krotova, Irina
and Panchenko, Alexander",
editor="Rosso, Paolo
and Basile, Valerio
and Mart{\'i}nez, Raquel
and M{\'e}tais, Elisabeth
and Meziane, Farid",
title="Studying the Role of Named Entities for Content Preservation in Text Style Transfer",
booktitle="Natural Language Processing and Information Systems",
year="2022",
publisher="Springer International Publishing",
address="Cham",
pages="437--448",
abstract="Text style transfer techniques are gaining popularity in Natural Language Processing, finding various applications such as text detoxification, sentiment, or formality transfer. However, the majority of the existing approaches were tested on such domains as online communications on public platforms, music, or entertainment yet none of them were applied to the domains which are typical for task-oriented production systems, such as personal plans arrangements (e.g. booking of flights or reserving a table in a restaurant). We fill this gap by studying formality transfer in this domain.",
isbn="978-3-031-08473-7"
}
```