<commit_before>DRISEE: Duplicate Read Inferred Sequencing Error Estimation
===

Contributors: <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, & <NAME>


DESCRIPTION
===

DRISEE is a tool that utilizes artifactual duplicate reads (ADRs) to provide a platform independent assessment 
of sequencing error in metagenomic (or genomic) sequencing data.  DRISEE is designed to consider shotgun data.
Currently, it is not appropriate for amplicon data. 


REQUIREMENTS
===

System:
---
A Unix environment ( Unix / Linux / OSX / Darwin / Cygwin / etc. ) 
DRISEE was developed to run on multicore Unix systems, but can be run on a desktop or laptop. DRISEE has been run on several OSX based systems.

Software:
---
- perl 5.12.1 or later (http://www.perl.org/get.html) *1
- python 2.6 or later (http://python.org/getit/)
		with biopython (http://biopython.org/wiki/Biopython) *1
- QIIME and the supporting uclust packages  (http://qiime.org/install/virtual_box.html) *1
- cdbfasta & cdbyank (http://sourceforge.net/projects/cdbfasta/) *1

*1 Please note that users are responsible for making sure that their use of required software
   products is compliant with existing licenses and/or user agreements. A full installation
   procedure for all requirements is provided below.  

FULL INSTALLATION
===

Note: This installation procedure has be tested on multiple ubuntu releases. It has not been rigorously tested on other platforms
---


CHANGE TEMP DIRECTORY TO SOMEPLACE WITH MORE SPACE (May not be necessary for your configuration, 10Gb free for DRISEE to use)

	# first remove the existing temp directory
	sudo rm -rf /tmp
	# create a directory that has more space
	sudo mkdir /mnt/my_temp_dir
	# then create symbolic link to the newly created directory  
	sudo ln -s /mnt/my_temp_dir /tmp
	# open permissions
	sudo chmod -R 777 /mnt/


INSTALL qiime_deploy and R dependencies

	# move to home directory
	cd ~
	# udpate
	sudo apt-get update
	# upgrade
	sudo apt-get upgrade 
	# install all of the dependencies
	sudo apt-get --force-yes -y install python-dev libncurses5-dev libssl-dev libzmq-dev libgsl0-dev openjdk-6-jdk libxml2 libxslt1.1 libxslt1-dev ant git subversion build-essential zlib1g-dev libpng12-dev libfreetype6-dev mpich2 libreadline-dev gfortran unzip libmysqlclient18 libmysqlclient-dev ghc sqlite3 libsqlite3-dev libc6-i386 libbz2-dev libx11-dev libcairo2-dev libcurl4-openssl-dev libglu1-mesa-dev freeglut3-dev mesa-common-dev xorg openbox emacs r-cran-rgl xorg-dev

---

INSTALL Qiime (1.8.0) # from https://github.com/qiime/qiime-deploy
---

Uncomment the universe and multiverse repositories from /etc/apt/sources.list

clone the deploy and configuration repos, and then perform the installation:

	# move to home directory
	cd ~
	# clone the qiime deploy repo
	git clone git://github.com/qiime/qiime-deploy.git
	# clone the qiime deploy configuration repo
	git clone git://github.com/qiime/qiime-deploy-conf.git
	# install qiime, in this case using the default configuration for 1.8.0
	sudo python ~/qiime-deploy/qiime-deploy.py ~/qiime_software/ -f ~/qiime-deploy-conf/qiime-1.8.0/qiime.conf --force-remove-failed-dirs
	# source the activation script; among other things, this will add everything that qiime has installed to your PATH variable
	source /qiime_software/activate.sh

---

INSTALL DRISEE (CLONE THE DRISEE REPOSITORY)

	# move to home directory
	cd ~
	# clone the DRISEE repository
	git clone https://github.com/MG-RAST/DRISEE.git

---

Add qiime and DRISEE to the PATH of the envrionment

	# source the activation script; among other things, this will add everything that qiime has installed to your PATH variable
	source ~/qiime/activate.sh
	# add the DRISEE directory to the PATH variable
	export PATH=$PATH:~/DRISEE
	# NOTE: you can add the path info for DRISEE and Qiime to your .profile and/or .bashrc so it will load every time you log into your machine

---

DESCRIPTION OF MAIN DRISEE SCRIPTS
===
       drisee.py           		(the main driving script - a main script file)
       seq_length_stats.py 		(an accessory script - generates sequence related stats)
       run_find_steiner.pl 		(an accessory script - performs iterative consensus sequence construction)
       qiime-uclust				(an accessory script - runs qiime-integrated uclust)
       
RUNNING DRISEE
===
DRISEE utilizes three scripts to perform its analyses.  All options and input parameters are passed to 
the main script (drisee.py) which calls the other scripts as needed. Each of the options and input 
parameters are explained below.  Examples of a typical execution are provided further below:
____
USAGE:

	drisee.py [options] input_seq_file output_stat_file_pattern
____
Input/Output summary:

	Input:  fasta/fastq file     (input_seq_file)
	Output: error matrix file(s) (output_stat_file_pattern)
	STDOUT: Runtime summary stats
____
Options:
<commit_msg>updated readme, Installation instructions<commit_after>DRISEE: Duplicate Read Inferred Sequencing Error Estimation
===

Contributors: <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, & <NAME>


DESCRIPTION
===

DRISEE is a tool that utilizes artifactual duplicate reads (ADRs) to provide a platform independent assessment 
of sequencing error in metagenomic (or genomic) sequencing data.  DRISEE is designed to consider shotgun data.
Currently, it is not appropriate for amplicon data. 


REQUIREMENTS
===

System:
---
A Unix environment ( Unix / Linux / OSX / Darwin / Cygwin / etc. ) 
DRISEE was developed to run on multicore Unix systems, but can be run on a desktop or laptop. DRISEE has been run on several OSX based systems.

Software:
---
- perl 5.12.1 or later (http://www.perl.org/get.html) *1
- python 2.6 or later (http://python.org/getit/)
		with biopython (http://biopython.org/wiki/Biopython) *1
- QIIME and the supporting uclust packages  (http://qiime.org/install/virtual_box.html) *1
- cdbfasta & cdbyank (http://sourceforge.net/projects/cdbfasta/) *1

*1 Please note that users are responsible for making sure that their use of required software
   products is compliant with existing licenses and/or user agreements. A full installation
   procedure for all requirements is provided below.  

FULL INSTALLATION
===

Note: This installation procedure has be tested on multiple ubuntu releases. It has not been rigorously tested on other platforms
---


CHANGE TEMP DIRECTORY TO SOMEPLACE WITH MORE SPACE (May not be necessary for your configuration, 10Gb free for DRISEE to use)

	# first remove the existing temp directory
	sudo rm -rf /tmp
	# create a directory that has more space
	sudo mkdir /mnt/my_temp_dir
	# then create symbolic link to the newly created directory  
	sudo ln -s /mnt/my_temp_dir /tmp
	# open permissions
	sudo chmod -R 777 /mnt/


INSTALL qiime_deploy and R dependencies

	# move to home directory
	cd ~
	# udpate
	sudo apt-get update
	# upgrade
	sudo apt-get upgrade 
	# install all of the dependencies
	sudo apt-get --force-yes -y install python-dev libncurses5-dev libssl-dev libzmq-dev libgsl0-dev openjdk-6-jdk libxml2 libxslt1.1 libxslt1-dev ant git subversion build-essential zlib1g-dev libpng12-dev libfreetype6-dev mpich2 libreadline-dev gfortran unzip libmysqlclient18 libmysqlclient-dev ghc sqlite3 libsqlite3-dev libc6-i386 libbz2-dev libx11-dev libcairo2-dev libcurl4-openssl-dev libglu1-mesa-dev freeglut3-dev mesa-common-dev xorg openbox emacs r-cran-rgl xorg-dev

---

INSTALL Qiime (1.8.0) # from https://github.com/qiime/qiime-deploy
---

Uncomment the universe and multiverse repositories from /etc/apt/sources.list

clone the deploy and configuration repos, and then perform the installation:

	# move to home directory
	cd ~
	# clone the qiime deploy repo
	git clone git://github.com/qiime/qiime-deploy.git
	# clone the qiime deploy configuration repo
	git clone git://github.com/qiime/qiime-deploy-conf.git
	# install qiime, in this case using the default configuration for 1.8.0
	sudo python ~/qiime-deploy/qiime-deploy.py ~/qiime_software/ -f ~/qiime-deploy-conf/qiime-1.8.0/qiime.conf --force-remove-failed-dirs
	# source the activation script; among other things, this will add everything that qiime has installed to your PATH variable
	source /qiime_software/activate.sh

---

INSTALL DRISEE (CLONE THE DRISEE REPOSITORY)

	# move to home directory
	cd ~
	# clone the DRISEE repository
	git clone https://github.com/MG-RAST/DRISEE.git

---

Add qiime and DRISEE to the PATH of the envrionment

	# source the activation script; among other things, this will add everything that qiime has installed to your PATH variable
	source ~/qiime_software/activate.sh
	# add the DRISEE directory to the PATH variable
	export PATH=$PATH:~/DRISEE
	# NOTE: you can add the path info for DRISEE and Qiime to your .profile and/or .bashrc so it will load every time you log into your machine

---

DESCRIPTION OF MAIN DRISEE SCRIPTS
===
       drisee.py           		(the main driving script - a main script file)
       seq_length_stats.py 		(an accessory script - generates sequence related stats)
       run_find_steiner.pl 		(an accessory script - performs iterative consensus sequence construction)
       qiime-uclust				(an accessory script - runs qiime-integrated uclust)
       
RUNNING DRISEE
===
DRISEE utilizes three scripts to perform its analyses.  All options and input parameters are passed to 
the main script (drisee.py) which calls the other scripts as needed. Each of the options and input 
parameters are explained below.  Examples of a typical execution are provided further below:
____
USAGE:

	drisee.py [options] input_seq_file output_stat_file_pattern
____
Input/Output summary:

	Input:  fasta/fastq file     (input_seq_file)
	Output: error matrix file(s) (output_stat_file_pattern)
	STDOUT: Runtime summary stats
____
Options:

{'fraction_non_alphanumeric': 0.07344979058126469, 'fraction_numerical': 0.010624169986719787, 'mean_word_length': 3.8803589232303093, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2511841', 'n_tokens_mistral': 3212, 'n_tokens_neox': 2964, 'n_words': 1175}