size_categories:1K<n<10K language:en license:mit arxiv:2308.04430 region:us
---
license: mit
language:
- en
size_categories:
- 1K<n<10K
dataset_info:
  features:
  - name: input
    dtype: string
  - name: label
    dtype: int64
  splits:
  - name: WikiMIA_length32
    num_bytes: 162091
    num_examples: 776
  - name: WikiMIA_length64
    num_bytes: 221018
    num_examples: 542
  - name: WikiMIA_length128
    num_bytes: 205118
    num_examples: 250
  - name: WikiMIA_length256
    num_bytes: 134879
    num_examples: 82
  download_size: 465221
  dataset_size: 723106
---


# 📘 WikiMIA Datasets

The **WikiMIA datasets** serve as a benchmark designed to evaluate membership inference attack (MIA) methods, specifically in detecting pretraining data from extensive large language models.

### 📌 Applicability

The datasets can be applied to various models released between **2017 to 2023**:

- LLaMA1/2
- GPT-Neo
- OPT
- Pythia
- text-davinci-001
- text-davinci-002
- ... and more.

## Loading the datasets

To load the dataset:

```python
from datasets import load_dataset

LENGTH = 64
dataset = load_dataset("swj0419/WikiMIA", split=f"WikiMIA_length{LENGTH}")
```
* Available Text Lengths: `32, 64, 128, 256`.
* *Label 0*: Refers to the unseen data during pretraining. *Label 1*: Refers to the seen data.

## 🛠️ Codebase

For evaluating MIA methods on our datasets, visit our [GitHub repository](https://github.com/swj0419/detect-pretrain-code).

## ⭐ Citing our Work

If you find our codebase and datasets beneficial, kindly cite our work:

```bibtex
@article{ ...,
    title={ .... },
    author={... },
    year={ 2023 },
    journal={ arXiv preprint arXiv:2308.04430 },
    url={ https://arxiv.org/abs/2308.04430 }
}
```



