<commit_before># Quick Start

A flowchart illustrating how each workflow component fits 
together with tools into the overall process is included below:

![workflow flowchart](img/WorkflowFlowchartOriginal.png)


## Quick Start: Read Filtering

To run the read filtering workflow, call snakemake and pass the
`read_filtering_workflow` build rule:

```
$ SINGULARITY_BINDPATH="data:/data" \
    snakemake \
    --use-singularity \
    read_filtering_workflow
``` 

When this command is run, it assembles the filenames of all files 
that are created by the post-trimming step of the read filtering
workflow.



More information about the read filtering workflow:

* [Read Filtering Walkthrough](readfilt_walkthru.md)
* [Read Filtering Snakemake Rules](readfilt_snakemake.md)
* [Read Filtering Parameters and Configuration](readfilt_config.md)


<commit_msg>add taxonomic classification section to quickstart<commit_after># Quick Start

A flowchart illustrating how each workflow component fits 
together with tools into the overall process is included below:

![workflow flowchart](img/WorkflowFlowchartOriginal.png)


## Setting a Scratch Directory

To set the directory in which all files will be placed, set the `data_dir`
key in the Snakemake configuration dictionary. For example, if the following
Snakemake configuration dictionary is in `params/custom_scratch.settings`, 
it will put all intermediate files into the `data/` directory:

**`params/custom_scratch.settings`:**

```
{
    "data_dir" : "data"
}
```

This can be used by passing the `--configfile` flag to Snakemake:

```
$ snakemake --configfile=params/custom_scratch.settings ...
```


<br />
<br />


## Quick Start: Read Filtering

The read filtering workflow requires that the user specify
the following:

* List of read files and corresponding URLs;
* List of samples to run the read filtering workflow on;
* List of quality values to use when performing read filtering;
* Pre-trimming and post-trimming filename patterns;
* Adapter file name and URL

These can be specified in a configuration file in JSON format
as follows:

**`params/read_filtering.config`:**

```
{
    #
    # Specify where to get the read files
    # 
    "files" : {
        #
        # URLs for full and subsampled reads
        "SRR606249_1_reads.fq.gz" :           "files.osf.io/v1/resources/dm938/providers/osfstorage/59f0f9156c613b026430dbc7",
        "SRR606249_2_reads.fq.gz" :           "files.osf.io/v1/resources/dm938/providers/osfstorage/59f0fc7fb83f69026076be47",
        "SRR606249_subset10_1_reads.fq.gz" :  "files.osf.io/v1/resources/dm938/providers/osfstorage/59f10134b83f69026377611b",
        "SRR606249_subset10_2_reads.fq.gz" :  "files.osf.io/v1/resources/dm938/providers/osfstorage/59f101f26c613b026330e53a",
        "SRR606249_subset25_1_reads.fq.gz" :  "files.osf.io/v1/resources/dm938/providers/osfstorage/59f1039a594d900263120c38",
        "SRR606249_subset25_2_reads.fq.gz" :  "files.osf.io/v1/resources/dm938/providers/osfstorage/59f104ed594d90026411f486",
        "SRR606249_subset50_1_reads.fq.gz" :  "files.osf.io/v1/resources/dm938/providers/osfstorage/59f1082d6c613b026430e5cf",
        "SRR606249_subset50_2_reads.fq.gz" :  "files.osf.io/v1/resources/dm938/providers/osfstorage/59f10ac6594d900262123e77",
    },


    #
    # Specify how to do read filtering
    # 
    "read_filtering" : {
        # 
        # The workflow actually builds the rules to download 
        # the read_files by using the pre_trimming_pattern.
        "read_patterns" : {
            #
            # filename pattern for pre-trimmed reads
            # Note: the files section (top) listing URLS for read files
            # MUST match the pre_trimmming_pattern.
            "pre_trimming_pattern"  : "{sample}_{direction}_reads.fq.gz",
            #
            # filename pattern for post-trimmed reads
            "post_trimming_pattern" : "{sample}_{direction}.trim{qual}.fq.gz",
        },
        #
        # Set the read adapter file
        "adapter_file" : {
            # 
            # name and URL for the sequencer adapter file
            "name" : "TruSeq2-PE.fa",
            "url"  : "http://dib-training.ucdavis.edu.s3.amazonaws.com/mRNAseq-semi-2015-03-04/TruSeq2-PE.fa"
        }
    },


    #
    # Specify what reads to filter
    # 
    "workflows" : {

        "read_filtering_workflow" : {
            #
            # These parameters determine which samples
            # the read filtering workflow will be run on.
            "sample"    : ["SRR606249_subset10","SRR606249_subset25"],
            "direction" : ["1","2"],
            "quality"   : ["2","30"],
        }
    }
}
```

To run the read filtering workflow, call snakemake and pass the
`read_filtering_workflow` build rule:

```
$ SINGULARITY_BINDPATH="data:/data" \
    snakemake \
    --use-singularity \
    read_filtering_workflow
``` 

When this command is run, it assembles the filenames of all files 
that are created by the post-trimming step of the read filtering
workflow.

More information about the read filtering workflow:

* [Read Filtering Walkthrough](readfilt_walkthru.md)
* [Read Filtering Snakemake Rules](readfilt_snakemake.md)
* [Read Filtering Parameters and Configuration](readfilt_config.md)


<br />
<br />


## Quick Start: Taxonomic Classification

The taxonomic classification workflow builds on the read filtering
workflow, so the user should specify all of the information
listed in the "Quick Start: Read Filtering" section (above). 
In addition, the user should specify the following:

* Percent threshold to use for taxa filtering
* Taxonomic rank to use for kaiju2krona
* Kaiju database name and URL









{'fraction_non_alphanumeric': 0.09121265377855887, 'fraction_numerical': 0.049033391915641475, 'mean_word_length': 3.2725225225225225, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2925724', 'n_tokens_mistral': 2041, 'n_tokens_neox': 1803, 'n_words': 490}