task_categories:text-classification task_categories:text-generation size_categories:100K<n<1M language:zh license:apache-2.0 novel training region:us
---
license: apache-2.0
language:
- zh
tags:
- novel
- training
task_categories:
- text-classification
- text-generation
pretty_name: CNNovel125K
size_categories:
- 100K<n<1M
duplicated_from: RyokoAI/CNNovel125K
---

# Dataset Card for CNNovel125K

*The BigKnow2022 dataset and its subsets are not yet complete. Not all information here may be accurate or accessible.*

## Dataset Description

- **Homepage:** (TODO)
- **Repository:** <https://github.com/RyokoAI/BigKnow2022>
- **Paper:** N/A 
- **Leaderboard:** N/A
- **Point of Contact:** Ronsor/undeleted <ronsor@ronsor.com>

### Dataset Summary

CNNovel125K is a dataset composed of approximately 125,000 novels downloaded from the Chinese novel hosting site <http://ibiquw.com>.

### Supported Tasks and Leaderboards

This dataset is primarily intended for unsupervised training of text generation models; however, it may be useful for other purposes.

* text-classification
* text-generation

### Languages

* Simplified Chinese

## Dataset Structure

### Data Instances

```json
{
  "text": "\n------------\n\n全部章节\n\n\n------------\n\n第一章 她肯定做梦呢！\n\n    HT国际大酒店总统套房。\n\n    清晨的第一缕阳光照射进圣地亚哥地板上，洒落在凌乱的床单上，突然地，床上睡的正熟的人睁开眼睛，
猛然惊醒！\n\n    ...",
  "meta": {
    "subset": "cnnovel.ibiquw",
    "id": "100067",
    "q": 0.9,
    "lang": "zh_cn",
    "title": "为爱入局：嫁给秦先生",
    "author": "奥德萨"
  }
}
{
  "text": "\n------------\n\n全部章节\n\n\n------------\n\n第1章：出狱就大婚\n\n    凉城第一监狱，大门缓缓打开，秦峰仰起头，贪婪的呼吸了一口空气。\n\n    三年了，终于又闻到了自由的味道。\n\n    他回过头，看着目
送他出来的那群人道：...",
  "meta": {
    "subset": "cnnovel.ibiquw",
    "id": "100059",
    "q": 0.9,
    "lang": "zh_cn",
    "title": "绝世弃婿",
    "author": "绷带怪"
  }
}
```

### Data Fields

* `text`: the actual novel text, all chapters
* `meta`: entry metadata
  * `subset`: dataset tag: `cnnovel.ibiquw`
  * `id`: novel ID
  * `q`: quality score, fixed at 0.9
  * `lang`: always `zh_cn` (Simplified Chinese)
  * `title`: novel title
  * `author`: novel author

### Data Splits

No splitting of the data was performed.

## Dataset Creation

### Curation Rationale

TODO

### Source Data

#### Initial Data Collection and Normalization

TODO

#### Who are the source language producers?

The authors of each novel.

### Annotations

#### Annotation process

Titles were collected alongside the novel text and IDs.

#### Who are the annotators?

There were no human annotators.

### Personal and Sensitive Information

The dataset contains only works of fiction, and we do not believe it contains any PII.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset is intended to be useful for anyone who wishes to train a model to generate "more entertaining" content in Chinese.
It may also be useful for other languages depending on your language model.

### Discussion of Biases

This dataset is composed of fictional works by various authors. Because of this fact, the contents of this dataset will reflect
the biases of those authors. Beware of stereotypes.

### Other Known Limitations

N/A

## Additional Information

### Dataset Curators

Ronsor Labs

### Licensing Information

Apache 2.0, for all parts of which Ronsor Labs or the Ryoko AI Production Committee may be considered authors. All other material is
distributed under fair use principles.

### Citation Information

```
@misc{ryokoai2023-bigknow2022,
  title         = {BigKnow2022: Bringing Language Models Up to Speed},
  author        = {Ronsor},
  year          = {2023},
  howpublished  = {\url{https://github.com/RyokoAI/BigKnow2022}},
}
```

### Contributions

Thanks to @ronsor (GH) for gathering this dataset.