license:apache-2.0 biology genomics medical region:us
---
tags:
- biology
- genomics
- medical
configs:
- config_name: demo_coding_vs_intergenomic_seqs
  data_files:
  - split: train
    path: demo_coding_vs_intergenomic_seqs/train/*.csv.gz
  - split: test
    path: demo_coding_vs_intergenomic_seqs/test/*.csv.gz
- config_name: demo_human_or_worm
  data_files:
  - split: train
    path: demo_human_or_worm/train/*.csv.gz
  - split: test
    path: demo_human_or_worm/test/*.csv.gz
- config_name: drosphilia_enhancers_stark
  data_files:
  - split: train
    path: drosphilia_enhancers_stark/train/*.csv.gz
  - split: test
    path: drosphilia_enhancers_stark/test/*.csv.gz
- config_name: dummy_mouse_enhancers_ensembl
  data_files:
  - split: train
    path: dummy_mouse_enhancers_ensembl/train/*.csv.gz
  - split: test
    path: dummy_mouse_enhancers_ensembl/test/*.csv.gz
- config_name: human_enhancers_cohn
  data_files:
  - split: train
    path: human_enhancers_cohn/train/*.csv.gz
  - split: test
    path: human_enhancers_cohn/test/*.csv.gz
- config_name: human_enhancers_ensembl
  data_files:
  - split: train
    path: human_enhancers_ensembl/train/*.csv.gz
  - split: test
    path: human_enhancers_ensembl/test/*.csv.gz
- config_name: human_ensembl_regulatory
  data_files:
  - split: train
    path: human_ensembl_regulatory/train/*.csv.gz
  - split: test
    path: human_ensembl_regulatory/test/*.csv.gz
- config_name: human_nontata_promoters
  data_files:
  - split: train
    path: human_nontata_promoters/train/*.csv.gz
  - split: test
    path: human_nontata_promoters/test/*.csv.gz
- config_name: human_ocr_ensembl
  data_files:
  - split: train
    path: human_ocr_ensembl/train/*.csv.gz
  - split: test
    path: human_ocr_ensembl/test/*.csv.gz
license: apache-2.0
---
# Genomic Benchmark
In this repository, we collect benchmarks for classification of genomic sequences. It is shipped as a Python package, together with functions helping to download & manipulate datasets and train NN models. 

## Citing Genomic Benchmarks

If you use Genomic Benchmarks in your research, please cite it as follows.

### Text

GRESOVA, Katarina, et al. Genomic Benchmarks: A Collection of Datasets for Genomic Sequence Classification. bioRxiv, 2022.

### BibTeX

```bib
@article{gresova2022genomic,
  title={Genomic Benchmarks: A Collection of Datasets for Genomic Sequence Classification},
  author={Gresova, Katarina and Martinek, Vlastimil and Cechak, David and Simecek, Petr and Alexiou, Panagiotis},
  journal={bioRxiv},
  year={2022},
  publisher={Cold Spring Harbor Laboratory},
  url={https://www.biorxiv.org/content/10.1101/2022.06.08.495248}
}
```

From the [github repo](https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks/tree/main):

# Datasets

Each folder contains either one benchmark or a set of benchmarks. See [docs/](../docs/) for code used to create these benchmarks.

### Naming conventions

* *dummy_...*: small datasets, used for testing purposes
* *demo_...*: middle size datasets, not necesarily biologically relevant or fully reproducible, used in demos

### Versioning

We recommend to check the version number when working with the dataset (i.e. not using default `None`). The version should be set to 0 when the dataset is proposed, after inicial curration it should be changed to 1 and then increased after every modification.

### Data format

Each benchmark should contain `metadata.yaml` file with its main folder with the specification in YAML format, namely

   * **the version** of the benchmark (0 = in development)

   * **the classes** of genomic sequences, for each class we further need to specify

       - *url* with the reference 
       - *type* of the reference (currently, only fa.gz implemented)
       - *extra_processing*, a parameter helping to overcome some know issues with identifiers matching

The main folder should also contain two folders, `train` and `test`. Both those folders should contain gzipped CSV files, one for each class (named `class_name.csv.gz`).

The format of gzipped CSV files closely resemble BED format, the column names must be the following:

* **id**: id of a sequence
* **region**: chromosome/transcript/... to be matched with the reference
* **start**, **end**: genomic interval specification (0-based, i.e. same as in Python)
* **strand**: either '+' or '-'


### To contribute a new datasets

Create a new branch. Add the new subfolders to `datasets` and `docs`. The subfolder of `docs` should contain a description of the dataset in `README.md`. If the dataset comes with the paper, link the paper. If the dataset is not taken from the paper, make sure you have described and understand the biological process behind it.

If you have access to `cloud_cache` folder on GDrive, upload your file there and update `CLOUD_CACHE` in [cloud_caching.py](https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks/blob/main/src/genomic_benchmarks/loc2seq/cloud_caching.py).

### To review a new dataset

Make sure you can run and reproduce the code. Check you can download the actual sequences and/or create a data loader. Do you understand what is behind these data? (either from the paper or the description) Ask for clarification if needed.