<commit_before>[![Build Status](https://travis-ci.com/biota/sourcetracker2.svg?token=c<PASSWORD>)](https://travis-ci.com/biota/sourcetracker2)

# SourceTracker2

SourceTracker was originally described in [Knights et al., 2011](http://www.ncbi.nlm.nih.gov/pubmed/21765408).
If you use this package, please cite the original SourceTracker paper linked 
above.

# Documentation

This script replicates and extends the functionality of <NAME>'s
SourceTracker R package.

The ``mapping file`` which describes the ``sources`` and ``sinks`` must be
formatted in the same way it was for the SourceTracker R package. Specifically, 
there must be a column ``SourceSink`` and a column ``Env``. For an example, look
at ``sourcetracker2/data/tiny-test/``. 

A major improvment in this version of SourceTracker is the ability to run it in parallel. 
Currently, parallelization across a single machine is
supported for both estimation of source proportions and leave-one-out source
class prediction. The speedup from parallelization should be approximately a 
factor of ``jobs`` that are passed. For instance, passing ``--jobs 4`` will
decrease runtime approximately 4X (less due to overhead). The package 
``ipyparallel`` is used to enable parallelization. Note that you can only specify 
as many jobs as you have sink samples. For instance, passing 10 jobs with only 5 
sink samples will not result in the code executing any faster than passing 5 jobs, 
since there is a 1 sink per job limit. Said another way, a single sink sample 
cannot be split up into multiple jobs.

# Installation
SourceTracker2 requires Python 3. If you don't have a local version of Python 3, you 
might want to install it using [Anaconda](https://docs.continuum.io/anaconda/install).

To install locally for SourceTracker2:

$cd /location/of/Anaconda.sh
$bash Anaconda.sh

Create the Python 3 environment named `py3` with required dependencies

$conda create -n py3 python=3 numpy scipy h5py hdf5

This will have created your local python 3 environment. To use your environment:

$source activate py3

Now we can get SourceTracker2 and install

$git clone https://github.com/biota/SourceTracker2.git
$python setup.py install

This should successfully install SourceTracker2 locally. Try calling:

$sourcetracker2 gibbs --help

# Theory

This readme describes some of the basic theory for use of SourceTracker2. For
more theory and a visual walkthrough, please see the [juypter notebook](https://github.com/biota/SourceTracker_rc/blob/master/ipynb/Sourcetracking%20using%20a%20Gibbs%20Sampler.ipynb).
<commit_msg>formatted readme<commit_after>[![Build Status](https://travis-ci.com/biota/sourcetracker2.svg?token=<PASSWORD>)](https://travis-ci.com/biota/sourcetracker2)

# SourceTracker2

SourceTracker was originally described in [Knights et al., 2011](http://www.ncbi.nlm.nih.gov/pubmed/21765408).
If you use this package, please cite the original SourceTracker paper linked 
above.

# Documentation

This script replicates and extends the functionality of Dan Knights's
SourceTracker R package.

The ``mapping file`` which describes the ``sources`` and ``sinks`` must be
formatted in the same way it was for the SourceTracker R package. Specifically, 
there must be a column ``SourceSink`` and a column ``Env``. For an example, look
at ``sourcetracker2/data/tiny-test/``. 

A major improvment in this version of SourceTracker is the ability to run it in parallel. 
Currently, parallelization across a single machine is
supported for both estimation of source proportions and leave-one-out source
class prediction. The speedup from parallelization should be approximately a 
factor of ``jobs`` that are passed. For instance, passing ``--jobs 4`` will
decrease runtime approximately 4X (less due to overhead). The package 
``ipyparallel`` is used to enable parallelization. Note that you can only specify 
as many jobs as you have sink samples. For instance, passing 10 jobs with only 5 
sink samples will not result in the code executing any faster than passing 5 jobs, 
since there is a 1 sink per job limit. Said another way, a single sink sample 
cannot be split up into multiple jobs.

# Installation
SourceTracker2 requires Python 3. If you don't have a local version of Python 3, you 
might want to install it using [Anaconda](https://docs.continuum.io/anaconda/install).

To install locally for SourceTracker2:

$cd /location/of/Anaconda.sh

$bash Anaconda.sh

Create the Python 3 environment named `py3` with required dependencies

$conda create -n py3 python=3 numpy scipy h5py hdf5

This will have created your local python 3 environment. To use your environment:

$source activate py3

Now we can get SourceTracker2 and install

$git clone https://github.com/biota/SourceTracker2.git

$python setup.py install

This should successfully install SourceTracker2 locally. Try calling:

$sourcetracker2 gibbs --help

# Theory

This readme describes some of the basic theory for use of SourceTracker2. For
more theory and a visual walkthrough, please see the [juypter notebook](https://github.com/biota/SourceTracker_rc/blob/master/ipynb/Sourcetracking%20using%20a%20Gibbs%20Sampler.ipynb).

{'fraction_non_alphanumeric': 0.07285409804301492, 'fraction_numerical': 0.018601046308854872, 'mean_word_length': 4.832768361581921, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 10, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '272554', 'n_tokens_mistral': 1526, 'n_tokens_neox': 1431, 'n_words': 654}