license:apache-2.0 region:us
---
license: 
- apache-2.0
---

This dataset fit on the streaming mode. 

The origin dataset link: https://data.commoncrawl.org/crawl-data/CC-MAIN-2022-27/warc.paths.gz  

_Requirements: selectolax, warcio_

```
from datasets import load_dataset

# sub name is the number has string type e.g. "1", "2", ...(it depends on the dataset)
dataset = load_dataset("psyche/common_crawl", "1", streaming=True)
```
