task_categories:text-classification language:en license:apache-2.0 region:us
---
license: apache-2.0
task_categories:
- text-classification
language:
- en
pretty_name: section 5 zst datasets
---
# Hugging Face course section 5 .zst datasets
You can use [these datasets](https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/tree/main/data) for whatever you want (note the [Apache 2.0 license](https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/blob/main/data/Apache_2.0), though) but their primary purpose is to serve as a drop-in replacement for the sub-datasets of [The Pile](https://pile.eleuther.ai/) used in [section 5](https://huggingface.co/learn/nlp-course/chapter5/4?fw=pt#what-is-the-pile) of the [HuggingFace course](https://huggingface.co/learn/nlp-course/chapter5/4?fw=pt#what-is-the-pile).

## Data sources
- PubMed-200k-RTC:<br>https://www.kaggle.com/datasets/matthewjansen/pubmed-200k-rtc/download?datasetVersionNumber=5
- LegalText-classification:<br>https://www.kaggle.com/datasets/shivamb/legal-citation-text-classification/download?datasetVersionNumber=1

These are Kaggle datasets. So you need to be logged into a [Kaggle account](https://www.kaggle.com/account/login?phase=startSignInTab&returnUrl=%2F) to download them from Kaggle. However, you actually don't need to download (and preprocess) them from Kaggle – you can just use them as shown in the following **Usage** section.

## Usage
To load a dataset from this repo, run

```python
import zstandard
from datasets import load_dataset
load_dataset("json", data_files=url, split="train")
```

where `url` should be one of the following download links:
- `LegalText-classification_train.jsonl.zst`:<br>https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/resolve/main/data/LegalText-classification_train.jsonl.zst,
- `LegalText-classification_train_min.jsonl.zst`:<br>https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/resolve/main/data/LegalText-classification_train_min.jsonl.zst,
- `PubMed-200k-RTC_train.jsonl.zst`:<br>https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/resolve/main/data/PubMed-200k-RTC_train.jsonl.zst, or
- `PubMed-200k-RTC_train_min.jsonl.zst`:<br>https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/resolve/main/data/PubMed-200k-RTC_train_min.jsonl.zst.

Example:

```python
import zstandard
from datasets import load_dataset
url = "https://huggingface.co/datasets/mdroth/PubMed-200k-RTC/resolve/main/data/LegalText-classification_train_min.jsonl.zst"
load_dataset("json", data_files=url, split="train")
```