multilinguality:monolingual size_categories:n<1K language:en license:other arxiv:2305.15002 region:us
Named-entities Relation Ranking.
TBA
---
language:
- en
license:
- other
multilinguality:
- monolingual
size_categories:
- n<1K
pretty_name: relentless
---

# Dataset Card for "cardiffnlp/relentless"

***RelEntLess*** is a new benchmark, in which entity pairs have to be ranked according to how much they satisfy a given graded relation.
Essentially, the task is a ranking task where we provide five prototypical examples to each relation. Following brief description of each relation type 
is used in our baseline in addition to the prototypical examples.
Please check our paper "[A RelEntLess Benchmark for Modelling Graded Relations between Named Entities](https://arxiv.org/abs/2305.15002)" for more detail.

```python
{
    "friend/ally of": "entities that are friends or allies",
    "competitor/rival of": "entities that are competitors or rivals",
    "known for": "examples of what entities are known for",
    "influenced by": "what has influenced different entities",
    "similar to": "examples of entities that are similar"
}
```




## Dataset Description
- **Repository:** [https://huggingface.co/datasets/cardiffnlp/relentless](https://huggingface.co/datasets/cardiffnlp/relentless)
- **Paper:** [A RelEntLess Benchmark for Modelling Graded Relations between Named Entities](https://arxiv.org/abs/2305.15002)
- **Dataset:** [https://huggingface.co/datasets/cardiffnlp/relentless](https://huggingface.co/datasets/cardiffnlp/relentless)

### Dataset Summary


| relation_type       |   val. |   test |
|:--------------------|-------:|-------:|
| competitor/rival of |     20 |     84 |
| friend/ally of      |     19 |     88 |
| influenced by       |     19 |     90 |
| known for           |     18 |    105 |
| similar to          |     19 |     89 |

## Dataset Structure
### Data Instances

```python
{
    "pairs": [["Le Corbusier", "purism art"], ["Sean Connery", "Finding Forrester"], ...],
    "scores_all": [[4.0, 5.0, 3.0, 4.0, 5.0, 3.0, 5.0], [4.0, 5.0, 2, 5.0, 5.0, 4.0, 2], ...],
    "scores_mean": [4.142857142857143, 3.857142857142857, 4.857142857142857, ...],
    "relation_type": "known for",
    "ranks": [8.5, 11, 5, 14, 15, 5, 20, 13, 1.5, 18, 10, 1.5, 17, ...],
    "prototypical_examples": [ [ "Russell Crowe", "Gladiator" ], [ "Cadbury", "chocolate" ],...]
}
```

### Citation Information
```
@misc{ushio2023relentless,
      title={A RelEntLess Benchmark for Modelling Graded Relations between Named Entities}, 
      author={Asahi Ushio and Jose Camacho Collados and Steven Schockaert},
      year={2023},
      eprint={2305.15002},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

