task_categories:text-classification task_ids:text-scoring task_ids:natural-language-inference task_ids:semantic-similarity-scoring source_datasets:assin2 language:pt region:us
---
language:
- pt
source_datasets:
- assin2
task_categories:
- text-classification
task_ids:
- text-scoring
- natural-language-inference
- semantic-similarity-scoring
---



# Dataset Card for ASSIN 2 (ADA)

### Dataset Summary

The ASSIN 2 corpus is composed of rather simple sentences. Following the procedures of SemEval 2014 Task 1.
The training and validation data are composed, respectively, of 6,500 and 500 sentence pairs in Brazilian Portuguese, 
annotated for entailment and semantic similarity. Semantic similarity values range from 1 to 5, and text entailment 
classes are either entailment or none. The test data are composed of approximately 3,000 sentence pairs with the same 
annotation. All data in the original dataset were manually annotated.

This dataset extends the original ASSIN2 adding the `cosine_similarity` column calculated using OpenAI's `text-embedding-ada-002` 
for research and benchmarking purposes, as it is currently considered one of the best multilingual models for this task.

### Languages

The language supported is Portuguese.

## Dataset Structure

### Data Fields

- `premise`: a `string` feature.
- `hypothesis`: a `string` feature.
- `relatedness_score`: a `float32` feature.
- `entailment_judgment`: a classification label, with possible values including `NONE`, `ENTAILMENT`.
- `ada_cosine_similarity`: the similarity calculated using OpenAI ada v2 embeddings

### Data Splits

The data is split into train, validation and test set. The split sizes are as follow:

| Train  | Val   | Test |
| ------ | ----- | ---- |
| 6500   | 500   | 2448 |

### Citation Information

```
@inproceedings{real2020assin,
  title={The assin 2 shared task: a quick overview},
  author={Real, Livy and Fonseca, Erick and Oliveira, Hugo Goncalo},
  booktitle={International Conference on Computational Processing of the Portuguese Language},
  pages={406--412},
  year={2020},
  organization={Springer}
}
```