<commit_before># TOIl-VG
## University of California, Santa Cruz Genomics Institute
### Please contact us on [github with any issues](https://github.com/BD2KGenomics/toil-vg/issues/new)

[vg](https://github.com/vgteam/vg) is a toolkit for DNA sequence analysis using variation graphs.  Toil-vg is a [toil](https://github.com/BD2KGenomics/toil)-based framework for running common vg pipelines at scale, either locally or on a distributed computing environment: 

`toil-vg construct`: Create vg graph from FASTA and VCF, constructing contigs in parallel.

`toil-vg run`: Given input vg graph(s), create indexes, map reads, then produce VCF variant calls.

`toil-vg index`: Produce a GCSA and/or XG index from input graph(s).

`toil-vg map`: Produce a graph alignment (GAM) for each chromosome from input reads and index

`toil-vg call`: Produce VCF from input XG index and GAM(s).

## Installation

### Local TOIL-VG Pip Installation

Installation requires Python and Toil.  We recommend installing within virtualenv as follows

    virtualenv toilvenv
    source toilvenv/bin/activate
    pip install toil[aws,mesos]==3.18.0
    pip install toil-vg
    
## WIKI

See the [Wiki](https://github.com/vgteam/toil-vg/wiki) in addition to below for examples.

### Docker

toil-vg can run vg, along with some other tools, via [Docker](http://www.docker.com).  Docker can be installed locally (not required when running via cgcloud), as follows. 
* [**Linux Docker Installation**](https://docs.docker.com/engine/installation/linux/): If running `docker version` doesn't work, try adding user to docker group with `sudo usermod -aG docker $USER`, then log out and back in.
* [**Mac Docker Installation**](https://docs.docker.com/docker-for-mac/): If running `docker version` doesn't work, try adding docker environment variables: `docker-machine start; docker-machine env; eval "$(docker-machine env default)"`
* **Running Without Docker**: If Docker is not installed or is disabled with `--container None`, toil-vg requires the following command line tools to be installed on the system: `vg, pigz, bcftools, tabix`.  `jq`, `samtools`, `rtg vcfeval`, 'hap.py', and 'Platypus.py' are also necessary for certain tests. 
    

## Configuration

A configuration file can be used as an alternative to most command line options.  A default configuration file can be generated using

    toil-vg generate-config > config.yaml

Pass this file to `toil-vg` commands using the `--config` option.

For non-trivial inputs, care must be taken to specify the resource requirements for the different pipeline phases (via the command line or by editing the config file), as they all default to single-core and 4G of ram.

To generate a default configuration for running at genome scale on a cluster with 32-core worker nodes, use

    toil-vg generate-config --whole_genome > config_wg.yaml

## Testing

To test toil-vg from a clean checkout, run:
    
    virutalenv venv
    . venv/bin/activate
    make prepare
    make develop
    make test
<commit_msg>Clarify that Docker support is Linux only<commit_after># TOIl-VG
## University of California, Santa Cruz Genomics Institute
### Please contact us on [github with any issues](https://github.com/BD2KGenomics/toil-vg/issues/new)

[vg](https://github.com/vgteam/vg) is a toolkit for DNA sequence analysis using variation graphs.  Toil-vg is a [toil](https://github.com/BD2KGenomics/toil)-based framework for running common vg pipelines at scale, either locally or on a distributed computing environment: 

`toil-vg construct`: Create vg graph from FASTA and VCF, constructing contigs in parallel.

`toil-vg run`: Given input vg graph(s), create indexes, map reads, then produce VCF variant calls.

`toil-vg index`: Produce a GCSA and/or XG index from input graph(s).

`toil-vg map`: Produce a graph alignment (GAM) for each chromosome from input reads and index

`toil-vg call`: Produce VCF from input XG index and GAM(s).

## Installation

### Local TOIL-VG Pip Installation

Installation requires Python and Toil.  We recommend installing within virtualenv as follows

    virtualenv toilvenv
    source toilvenv/bin/activate
    pip install toil[aws,mesos]==3.18.0
    pip install toil-vg
    
## WIKI

See the [Wiki](https://github.com/vgteam/toil-vg/wiki) in addition to below for examples.

### Docker

toil-vg can run vg, along with some other tools, via [Docker](http://www.docker.com).  Docker can be installed locally (not required when running via cgcloud), as follows. Docker is now only supported on Linux.  On Mac, all required executables must be installed locally.
* [**Linux Docker Installation**](https://docs.docker.com/engine/installation/linux/): If running `docker version` doesn't work, try adding user to docker group with `sudo usermod -aG docker $USER`, then log out and back in.
* **Running Without Docker**: If not running in Linux, Docker is not installed or Docker is disabled with `--container None`, toil-vg requires the following command line tools to be installed on the system: `vg, pigz, bcftools, tabix`.  `jq`, `samtools`, `rtg vcfeval`, 'hap.py', and 'Platypus.py' are also necessary for certain tests. 
    

## Configuration

A configuration file can be used as an alternative to most command line options.  A default configuration file can be generated using

    toil-vg generate-config > config.yaml

Pass this file to `toil-vg` commands using the `--config` option.

For non-trivial inputs, care must be taken to specify the resource requirements for the different pipeline phases (via the command line or by editing the config file), as they all default to single-core and 4G of ram.

To generate a default configuration for running at genome scale on a cluster with 32-core worker nodes, use

    toil-vg generate-config --whole_genome > config_wg.yaml

## Testing

To test toil-vg from a clean checkout, run:
    
    virutalenv venv
    . venv/bin/activate
    make prepare
    make develop
    make test

{'fraction_non_alphanumeric': 0.08726178535606821, 'fraction_numerical': 0.003009027081243731, 'mean_word_length': 4.122431506849315, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 11, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3415996', 'n_tokens_mistral': 1829, 'n_tokens_neox': 1686, 'n_words': 765}