<jupyter_start><jupyter_text>pysradb search The pysradb search module supports querying the Sequence Read Archive (SRA) and the European Nucleotide Archive (ENA) databases for sequencing data. The module also includes several built-in flags that can be used to fine-tune a search query.<jupyter_code>%%html
<style>
th {font-size: 16px;}
td {font-size: 14px;}
td:first-child {font-size: 15px; font-weight: 500;}
</style><jupyter_output><empty_output><jupyter_text>Terminal flags for the pysradb search module:|Flags | Explanation||----------|------------|| -h, --help | Displays the help message || --saveto | Saves the result in the file specified by the user.Supported file types: txt, tsv, csv || --db   | Selects the database (SRA, ENA, or both SRA and Geo DataSets) to query. Default database is SRA. Accepted inputs: sra, ena, geo|| -v, --verbosity  | This determines how much details are retrieved and shown in the search result: 0: run_accession only 1: run_accession and experiment_description only 2: (default) study_accession, experiment_accession, experiment_title, description, tax_id, scientific_name, library_strategy, library_source, library_selection, sample_accession, sample_title, instrument_model, run_accession, read_count, base_count 3: Everything in verbosity level 2, followed by all other retrievable information from the database|| -m, --max | Maximum number of returned entries. Default number is 20.Note: If the maximum number set is large, querying the SRA and GEO DataSets databases will take significantly longer due to API limits|| -q, --query | The main query string. Note: if this flag is not used, at least one of the following flags must be supplied: || --accession | A relevant study / experiment / sample / run accession number|| --organism | Scientific name of the sample organism || --layout | Library layout. Accepted inputs: single, paired|| --mbases | Size of the sample rounded to the nearest megabase|| --publication-date | The publication date of the run in the format dd-mm-yyyy. If a date range is desired, enter the start date, followed by end date, separated by a colon ':' in the format dd-mm-yyyy:dd-mm-yyyy Example: 01-01-2010:31-12-2010|| --platform | Sequencing platform used for the run. Possible inputs: illumina, ion torrent, oxford nanopore || --selection | Library selection. Possible inputs: cdna, chip, dnase, pcr, polya || --source | Library source. Possible inputs: genomic, metagenomic, transcriptomic || --strategy | Library Preparation strategy. Possible inputs: wgs, amplicon, rna seq || --title | Title of the experiment associated with the run || --geo-query | The main query string to be sent to Geo DataSets || --geo-dataset-type | Dataset type. Possible inputs: expression profiling by array, expression profiling by high throughput sequencing, non coding rna profiling by high throughput sequencing || --geo-entry-type | Entry type. Accepted inputs: gds, gpl, gse, gsm |  Using pysradb search in python: pysradb search organises each search query as a instance of either the SraSearch, EnaSearch or the GeoSearch classes. These classes takes in the following parameters in their constructor: `SraSearch (verbosity=2, return_max=20, query=None, accession=None, organism=None, layout=None, mbases=None, publication_date=None, platform=None, selection=None, source=None, strategy=None, title=None, suppress_validation=False,) ` `EnaSearch (verbosity=2, return_max=20, query=None, accession=None, organism=None, layout=None, mbases=None, publication_date=None, platform=None, selection=None, source=None, strategy=None, title=None, suppress_validation=False,) ` `GeoSearch (verbosity=2, return_max=20, query=None, accession=None, organism=None, layout=None, mbases=None, publication_date=None, platform=None, selection=None, source=None, strategy=None, title=None, geo_query=None, geo_dataset_type=None, geo_entry_type=None, suppress_validation=False,) ` | Parameters | Explanations||----------|------------|| verbosity | This determines how much details are retrieved and shown in the search result (default=2). Same as -v / --verbosity on terminal || return_max | Maximum number of returned entries (default=20). Same as -m / --max on terminal || suppress_validation | Defaults to False. If this is set to True, the user input format checks will be skipped. Setting this to True may cause the program to behave in unexpected ways, but allows the user to search queries that does not pass the format check.|Other parameters match the command line flags of the same name.  To query the SRA database for ribosome profiling, expecting an output of verbosity level 2, and returning at most 5 entries, we can do the following:<jupyter_code>from pysradb.search import SraSearch

instance = SraSearch(2, 5, query="ribosome profiling")
instance.search()
df = instance.get_df()
print(df)<jupyter_output>c:\users\lenovo\appdata\local\programs\python\python37\lib\site-packages\pysradb\utils.py:14: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
  from tqdm.autonotebook import tqdm
100%|██████████| 5/5 [00:01<00:00,  3.01it/s]<jupyter_text>Quickstart To query ENA instead, replace SraSearch class with the EnaSearch class:<jupyter_code>from pysradb.search import EnaSearch

instance = EnaSearch(2, 5, "ribosome profiling")
instance.search()
df = instance.get_df()
print(df)<jupyter_output>study_accession experiment_accession  \
0      PRJEB12126           ERX1264364   
1      PRJEB12126           ERX1264365   
2      PRJEB12126           ERX1264366   
3      PRJEB12126           ERX1264367   
4      PRJEB12126           ERX1264368   

                                    experiment_title  \
0  Illumina HiSeq 2000 sequencing; Analysis of co...   
1  Illumina HiSeq 2000 sequencing; Analysis of co...   
2  Illumina HiSeq 2000 sequencing; Analysis of co...   
3  Illumina HiSeq 2000 sequencing; Analysis of co...   
4  Illumina HiSeq 2000 sequencing; Analysis of co...   

                                         description tax_id scientific_name  \
0  Illumina HiSeq 2000 sequencing; Analysis of co...  10090    Mus musculus   
1  Illumina HiSeq 2000 sequencing; Analysis of co...  10090    Mus musculus   
2  Illumina HiSeq 2000 sequencing; Analysis of co...  10090    Mus musculus   
3  Illumina HiSeq 2000 sequencing; Analysis of co...  10090    Mus musculus   
4  Illumina HiS[...]<jupyter_text>To query GEO DataSets instead and retrieve the metadata of linked entries in SRA:<jupyter_code>from pysradb.search import GeoSearch

instance = GeoSearch(2, 5, geo_query="ribosome profiling")
instance.search()
df = instance.get_df()
print(df)<jupyter_output>100%|██████████| 5/5 [00:01<00:00,  3.29it/s]<jupyter_text>Error Handling When suppress_validation is not set to True, query fields with incorrect entries will raise IncorrectFieldException, which provides the complete list of acceptable inputs for fields such as "selection", etc:<jupyter_code># 1. Invalid query entered for "selection"
SraSearch(selection="Mudkip")
# 2. Ambiguous query entered for "source":
EnaSearch(source="metagenomic viral rna ")<jupyter_output><empty_output><jupyter_text>Usage Examples:  1. Checking the help message on terminal:<jupyter_code>!pysradb search -h<jupyter_output>usage: pysradb search [-h] [--saveto SAVETO] [--db {ena,geo,sra}]
                      [-v {0,1,2,3}] [-m MAX] [-q QUERY [QUERY ...]]
                      [--accession ACCESSION]
                      [--organism ORGANISM [ORGANISM ...]]
                      [--layout {SINGLE,PAIRED}] [--mbases MBASES]
                      [--publication-date PUBLICATION_DATE]
                      [--platform PLATFORM [PLATFORM ...]]
                      [--selection SELECTION [SELECTION ...]]
                      [--source SOURCE [SOURCE ...]]
                      [--strategy STRATEGY [STRATEGY ...]]
                      [--title TITLE [TITLE ...]]
                      [--geo-query GEO_QUERY [GEO_QUERY ...]]
                      [--geo-dataset-type GEO_DATASET_TYPE [GEO_DATASET_TYPE ...]]
                      [--geo-entry-type GEO_ENTRY_TYPE [GEO_ENTRY_TYPE ...]]

optional arguments:
  -h, --help            show this help message and exit
  --saveto SAVETO       Save search result datafram[...]<jupyter_text>2. Searching for 5 illumina sequences related to the covid-19 pandemic on ENA, using the terminal:<jupyter_code>!pysradb search -q covid-19 --platform illumina --db ena -m 5<jupyter_output>study_accession	experiment_accession	experiment_title	description	tax_id	scientific_name	library_strategy	library_source	library_selection	sample_accession	sample_title	instrument_model	run_accession	read_count	base_count

PRJNA612578  SRX7918726                                                                            Illumina MiSeq sequencing; RNA-seq of COVID-19 positive human  from San Diego county                                                                            Illumina MiSeq sequencing; RNA-seq of COVID-19 positive human  from San Diego county  2697049  Severe acute respiratory syndrome coronavirus 2  RNA-Seq           VIRAL RNA     PCR  SAMN14380341                                     Pathogen: clinical or host-associated sample from Severe acute respiratory syndrome coronavirus 2  Illumina MiSeq  SRR11314339   517281   156068315
 PRJNA612578  SRX7949318                                                                                     MinION sequencing; RNA-seq of C[...]<jupyter_text>3. Searching for illumina sequences related to the covid-19 pandemic on ENA, using the terminal, and saving the results in a nicely formatted text file:<jupyter_code>!pysradb search -q covid-19 --db ena --saveto query.txt<jupyter_output><empty_output><jupyter_text>4. Searching for illumina sequences related to the covid-19 pandemic on ENA, within python: (outputs a pandas dataframe)<jupyter_code>from pysradb.search import EnaSearch

instance = EnaSearch(2, 20, query="covid-19", platform="illumina")
instance.search()
df = instance.get_df()
print(df)<jupyter_output>study_accession experiment_accession  \
0      PRJNA612578           SRX7918726   
1      PRJNA612578           SRX7949318   
2      PRJNA613951           SRX7966547   
3      PRJNA624231           SRX8089022   
4      PRJNA625669           SRX8128264   
5      PRJNA628125           SRX8176770   
6      PRJNA628125           SRX8176769   
7      PRJNA628125           SRX8176768   
8      PRJNA628125           SRX8176767   
9      PRJNA628125           SRX8176766   
10     PRJNA628125           SRX8176765   
11     PRJNA628125           SRX8176764   
12     PRJNA628125           SRX8176763   
13     PRJNA628125           SRX8176762   
14     PRJNA628125           SRX8176761   
15     PRJNA628125           SRX8176760   
16     PRJNA628125           SRX8176759   
17     PRJNA628125           SRX8176758   
18     PRJNA628125           SRX8176757   
19     PRJNA632678           SRX8341156   

                                     experiment_title  \
0   Illumina MiSeq sequencing; RNA-seq [...]<jupyter_text>5. More complex example:<jupyter_code>from pysradb.search import EnaSearch

instance = EnaSearch(
    3, 100000, query="Escherichia coli", accession="SRS6898940", organism="Escherichia coli", 
    layout="Paired", mbases=10, publication_date="01-01-2019:31-12-2019", platform="Illumina", 
    selection="random", source="Genomic", strategy="WGS"
)
instance.search()
df = instance.get_df()
print(df)<jupyter_output>study_accession experiment_accession  \
0          PRJDA51849            DRX000357   
1              PRJDB5            DRX001133   
2           PRJDB2622            DRX001465   
3           PRJDB2622            DRX001466   
4           PRJDB2622            DRX001467   
...               ...                  ...   
37904     PRJNA230969           SRX6737267   
37905     PRJNA230969           SRX6737268   
37906     PRJNA368991           SRX6737270   
37907     PRJNA230969           SRX6737273   
37908     PRJNA318589           SRX6737281   

                                        experiment_title  \
0      Illumina Genome Analyzer sequencing; Deep sequ...   
1      AB SOLiD System 3.0 sequencing; Escherichia co...   
2      Illumina Genome Analyzer IIx sequencing; Reseq...   
3      Illumina Genome Analyzer IIx sequencing; Reseq...   
4      Illumina Genome Analyzer IIx sequencing; Reseq...   
...                                                  ...   
37904  Illumina MiSeq paire[...]<jupyter_text>6. Corresponding terminal command example, with max set to 20:<jupyter_code>!pysradb search --db ena -m 20 -v 3 -q Escherichia coli --accession SRS6898940 --organism Escherichia coli --layout Paired --mbases 100 --publication-date 01-01-2019:31-12-2019 --platform illumina --selection random --source Genomic --strategy wgs<jupyter_output>study_accession	experiment_accession	experiment_title	description	tax_id	scientific_name	library_strategy	library_source	library_selection	sample_accession	sample_title	instrument_model	run_accession	read_count	base_count	accession	altitude	assembly_quality	assembly_software	binning_software	bio_material	broker_name	cell_line	cell_type	center_name	checklist	collected_by	collection_date	completeness_score	contamination_score	country	cram_index_aspera	cram_index_ftp	cram_index_galaxy	cultivar	culture_collection	depth	dev_stage	ecotype	elevation	environment_biome	environment_feature	environment_material	environmental_package	environmental_sample	experiment_alias	experimental_factor	fastq_aspera	fastq_bytes	fastq_ftp	fastq_galaxy	fastq_md5	first_created	first_public	germline	host	host_body_site	host_genotype	host_gravidity	host_growth_conditions	host_phenotype	host_sex	host_status	host_tax_id	identified_by	instrument_platform	investigation_type	isolate	isolation_source	last_updated	lat	lib[...]
{'fraction_non_alphanumeric': 0.078205865439908, 'fraction_numerical': 0.059516963772282924, 'mean_word_length': 3.032753623188406, 'pattern_counts': {'":': 1, '<': 41, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '319650', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 4740, 'n_tokens_neox': 3990, 'n_words': 1327, 'path': 'notebooks/08.Query_Search.ipynb', 'repo_name': 'Phlya/pysradb', 'size': '13912'}