task_categories:text-classification task_categories:text-retrieval task_ids:natural-language-inference task_ids:document-retrieval multilinguality:monolingual size_categories:100K<n<1M source_datasets:fever language:cs license:cc-by-sa-3.0 Fact-checking arxiv:2201.11115 region:us
This new dataset is aimed on Czech fact-checking task.
---
license: cc-by-sa-3.0
task_categories:
- text-classification
- text-retrieval
task_ids:
- natural-language-inference
- document-retrieval
language:
- cs
tags:
- Fact-checking
pretty_name: CsFEVERv2
multilinguality: monolingual
source_datasets: fever
size_categories:
- 100K<n<1M
---
# Dataset Card for "CsFEVERv2"

## Dataset Description

CsFEVERv2 is a dataset for Czech fact-checking developed as part of a bachelor thesis at the Artificial Intelligence Center of the Faculty of Electrical Engineering of 
the Czech technical university in Prague. The dataset consists of an **original** subset, which is only an iteration of CsFEVER with new data and better processing and 
**f1**, **precision**, and **07** subsets filtered using an NLI model and optimized threshold values. The subset **wiki_pages** is a processed Wikipedia dump from 
August 2022 with correct revids. This subset should be used to map evidence from datasets to Wikipedia texts. Additionaly preprocessed datasets **original_nli**, **f1_nli**, **precision_nli**, **07_nli**,
for training of NLI models are included. 

The original subset can be used to generate other filtered datasets by filtering with other thresholds using predicted_label and predicted_score fields.

### Languages

Czech

## Dataset Usage Example

```python
from datasets import load_dataset

#load default (original) subset
dataset = load_dataset("/home/mlynatom/csfever_v2")
dataset = load_dataset("/home/mlynatom/csfever_v2", "original")

#load f1, f1_nli, precision, precision_nli, 07, and 07_nli subsets
dataset = load_dataset("/home/mlynatom/csfever_v2", "f1")

#load wiki_pages subset
dataset = load_dataset("/home/mlynatom/csfever_v2", "wiki_pages")

```

## Dataset Structure

### Data Instances

#### original
An example of 'train' looks as follows.
```json
{'id': 75397,
 'label': 'SUPPORTS',
 'predicted_label': 'SUPPORTS',
 'predicted_score': 0.921731
 'claim': 'Nikolaj Coster-Waldau pracoval pro Fox Broadcasting Company.',
 'evidence': [ [ "Nikolaj Coster-Waldau", "Nikolaj Coster-Waldau" ], [ "Fox Broadcasting Company", "Fox Broadcasting Company" ] ]}
```

#### f1, precision, 07
An example of 'train' looks as follows.
```json
{'id': 75397,
 'label': 'SUPPORTS',
 'claim': 'Nikolaj Coster-Waldau pracoval pro Fox Broadcasting Company.',
 'evidence': [ [ "Nikolaj Coster-Waldau", "Nikolaj Coster-Waldau" ], [ "Fox Broadcasting Company", "Fox Broadcasting Company" ] ]}
```

#### original_nli, f1_nli, precision_nli, 07_nli
An example of 'train' looks as follows.
```json
{'id': 155439,
 'label': 2,
 'claim': 'Newcastle United FC vyhrál pět ligových titulů.',
 'evidence': "Ronnie Simpson. Ronnie Simpson (21. října 1930, Glasgow – 19. dubna 2004, Edinburgh) byl skotský fotbalový brankář..."}
```

#### wiki_pages
An example of 'wiki_pages' looks as follows.
```json
{'id': 80916,
 'revid': 20561555,
 'url': "https://cs.wikipedia.org/wiki?curid=80916",
 'title': "Altruismus",
 'text': "Altruismus (z lat. "alter", druhý, 3. pád "altrui", druhému) je moderní ..."}
```

### Data Fields

#### original

- `id`: a `int32` feature.
- `label`: a `string` feature.
- `predicted_label`: a `string` feature. (label predicted by NLI model)
- `predicted_score`: a `int32` feature. (confidence of predicted_label predicted by NLI model)
- `claim`: a `string` feature.
- `evidence`: a `sequence` feature.

#### f1, precision, 07

- `id`: a `int32` feature.
- `label`: a `string` feature.
- `claim`: a `string` feature.
- `evidence`: a `sequence` feature.

#### original_nli, f1_nli, precision_nli, 07_nli

- `id`: a `int32` feature.
- `label`: a `int32` feature.
- `claim`: a `string` feature.
- `evidence`: a `string` feature.

#### wiki_pages
- `id`: a `int32` feature.
- `revid`: a `int32` feature.
- `url`: a `string` feature.
- `title`: a `string` feature.
- `text`: a `string` feature.


### Data Splits

### Data Splits
#### original
|          |  train | dev  | test  |
|----------|-------:|-----:|------:|
| original | 118950 | 7458 | 7520  |

#### f1

|    | train | dev  | test |
|----|------:|-----:|-----:|
| f1 | 83438 | 5445 | 5328 |

#### precision

|           |  train | dev  | test  |
|-----------|-------:|-----:|------:|
| precision |  60828 | 4288 |  4236 |

#### 07

|    |  train | dev  | test  |
|----|-------:|-----:|------:|
| 07 | 108607 | 6685 | 6623  |

#### wiki_pages
|            | wiki_pages |
|------------|-----------:|
| wiki_pages |    825078  |

# Citation

```bibtex
@article{Ullrich_2023,
	doi = {10.1007/s10579-023-09654-3},
	url = {https://doi.org/10.1007%2Fs10579-023-09654-3},
	year = 2023,
	month = {may},
	publisher = {Springer Science and Business Media {LLC}},
	author = {Herbert Ullrich and Jan Drchal and Martin Rýpar and Hana Vincourová and Václav Moravec},
	title = {{CsFEVER} and {CTKFacts}: acquiring Czech data for fact verification},
	journal = {Language Resources and Evaluation},
    archivePrefix={arXiv},
    eprint={2201.11115},
}
```

```bibtex
@thesis{Mlynar_2023,
  author      = {Mlynář, Tomáš},
  type        = {Bachelor's Thesis}
  title       = {Automated Fact Checking Based on Czech Wikipedia},
  institution = {Czech Technical University in Prague, Faculty of Electrical Engineering},
  date        = {2023},
  url         = {http://hdl.handle.net/10467/109219}
}
```
