region:us
# Human variants
A curated set of variants from three sources: ClinVar, COSMIC, OMIM and gnomAD.
Predictions for methods benchmarked in GPN-MSA paper can be [downloaded from here](https://huggingface.co/datasets/songlab/human_variants/resolve/main/variants_and_preds.parquet).
Functional annotations can be [downloaded from here](https://huggingface.co/datasets/songlab/human_variants/resolve/main/functional_annotations.zip).

## Data sources

**ClinVar**:
Missense variants considered "Pathogenic" by human labelers.

**COSMIC**:
Somatic missense variants with a frequency at least 0.1% in cancer samples (whole-genome and whole-exome sequencing only).

**OMIM**:
Regulatory variants considered "Pathogenic" by human labelers, curated in [this paper](https://doi.org/10.1016/j.ajhg.2016.07.005).

**gnomAD**:
All common variants (MAF > 5%) as well as an equally-sized subset of rare variants (MAC=1). Only autosomes are included.

## Usage
```python
from datasets import load_dataset

dataset = load_dataset("songlab/human_variants", split="test")
```

Subset - ClinVar Pathogenic vs. gnomAD common (missense) (can specify `num_proc` to speed up):
```python
dataset = dataset.filter(lambda v: v["source"]=="ClinVar" or (v["label"]=="Common" and "missense" in v["consequence"]))
```

Subset - COSMIC frequent vs. gnomAD common (missense):
```python
dataset = dataset.filter(lambda v: v["source"]=="COSMIC" or (v["label"]=="Common" and "missense" in v["consequence"]))
```

Subset - OMIM Pathogenic vs. gnomAD common (regulatory):
```python
cs = ["5_prime_UTR", "upstream_gene", "intergenic", "3_prime_UTR", "non_coding_transcript_exon"]
dataset = dataset.filter(lambda v: v["source"]=="OMIM" or (v["label"]=="Common" and "missense" not in v["consequence"] and any([c in v["consequence"] for c in cs])))
```

Subset - gnomAD rare vs. gnomAD common:
```python
dataset = dataset.filter(lambda v: v["source"]=="gnomAD")
```