task_categories:text-classification task_categories:text-generation size_categories:100K<n<1M language:en license:apache-2.0 training text region:us
---
license: apache-2.0
language:
- en
tags:
- training
- text
task_categories:
- text-classification
- text-generation
pretty_name: TvTroper
size_categories:
- 100K<n<1M
---

# Dataset Card for TvTroper

*TvTroper is a public raw dataset on TvTropes.org page.*

## Dataset Description

- **Homepage:** (TODO)
- **Repository:** N/A
- **Paper:** N/A
- **Leaderboard:** N/A
- **Point of Contact:** KaraKaraWitch

### Dataset Summary

TvTroper is a raw dataset dump consisting of text from at most 651,522 wiki pages (excluding namespaces and date-grouped pages) from tvtropes.org.

### Supported Tasks and Leaderboards

This dataset is primarily intended for unsupervised training of text generation models; however, it may be useful for other purposes.

- text-classification
- text-generation

### Languages

- English

## Dataset Structure

All the files are located in jsonl files that has been compressed into a 20GB .zip archive.

### Data Instances

```json
["https://tvtropes.org/pmwiki/pmwiki.php/HaruhiSuzumiya/TropesJToN","<!DOCTYPE html>\n\t<html>\n\t\t<head lang=\"en\">\n...<TRUNCATED>"]
```

### Data Fields

There is only 2 fields in the list. URL and content retrieved. Content retrieved may contain errors. If the page does not exist, the 404 error page is scraped.  

For the case of 1 specific URL: `https://tvtropes.org/pmwiki/pmwiki.php/JustForFun/RedirectLoop` will endlessly redirect to the page.  
As such we have used the following html as placeholder for such occurances:

```html
<!DOCTYPE html><html><head lang=\"en\"><title>Error: URL Exceeds maximum allowed redirects.</title></head><body class=\"\"><div>Error: URL Exceeds maximum allowed redirects.</div></body></html>
```

URLs may not match the final url in which the page was retrieved from. As they may be redirects present while scraping.

#### Q-Score Distribution

Not Applicable

### Data Splits

The jsonl files are split by their namespaces.

## Dataset Creation

### Curation Rationale

We have curated TvTropes.org as it serves as one of the best resource for common themes, narrative devices, and character archetypes that shape our various stories around the world.

### Source Data

#### Initial Data Collection and Normalization

None. No normalization is performed as this is a raw dump of the dataset.

#### Who are the source language producers?

The related editors/users of TvTropes.org

### Annotations

#### Annotation process

No Annotations are present.

#### Who are the annotators?

No human annotators.

### Personal and Sensitive Information

We are certain there is no PII included in the dataset.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset is intended to be useful for anyone who wishes to train a model to generate "more entertaining" content.
It may also be useful for other languages depending on your language model.

### Discussion of Biases

This dataset contains mainly TV Tropes used in media.

### Other Known Limitations

N/A

## Additional Information

### Dataset Curators

KaraKaraWitch

### Licensing Information

Apache 2.0, for all parts of which KaraKaraWitch may be considered authors. All other material is distributed under fair use principles.  

Ronsor Labs additionally is allowed to relicense the dataset as long as it has gone through processing.

### Citation Information

```
@misc{tvtroper,
  title         = {TvTroper: Tropes & Others.},
  author        = {KaraKaraWitch},
  year          = {2023},
  howpublished  = {\url{https://huggingface.co/datasets/RyokoExtra/TvTroper}},
}
```

### Name Etymology

N/A

### Contributions

- [@KaraKaraWitch (Twitter)](https://twitter.com/KaraKaraWitch) for gathering this dataset.
