<commit_before>"""
pipeline_peakcalling.py - Produce Peaklist from Bam Files 
===================================================

:Author: <NAME> & <NAME>
:Release: $Id$
:Date: |today|
:Tags: Python


Methods
=======

Pipeline Usage
=============
- Takes Bam files that you want to call peaks in and their
appropriate 'input' controls.
- Call peaks for bam files matching them to thier inputs. 
	- Produce Bed files containing peaks for downstram analysis 

	Optional functions: 
	-------------------
	- Filter Bam files to remove:
			- Duplicates
			- Secondary alignments
			- Unpaired reads for paired-end files
			- Reads overlapping 'blacklisted' regions 
			- Mapping quality (MAPQ) score
	- Pool input files for peakcalling to give better peakcalling
	  when inputs have poor coverage or lack of sequening depth 
	- Perform Irreproducible Discovery Rate (IDR) analysis (described 
	  further below) to get a consensus list of 
	  'highly reproducible peaks' and assess replicate quaility 


NOTE: WARNINGS!!!! 

1. IDR analysis may not be approprate for all type of peak file - It works 
best with transcription factor CHIPs or methodologies producing 'narrow peaks'
or peaks with well defined boundaries 
	
BroadPeak IDR might not work becuase peak boundary's are harder
to define and thus may not be so reproducible between replicates 

2. Always check your output from this pipeline in a genome browser to check 
peaks are being called suffiently!

3. This pipeline references ChIP bams throughout in the code -this 
referencces the immunoprecipitated (IP) sample from a ChIP experiment 
(i.e. the file you want to find peaks in), Input bams refer to the 
bams of the input control samples that are used for background 
normalisation in peak calling. Although we refer to ChIP bams 
this is only nomenclature and you could just as easily use
an ATAC-Seq bam file or other bam file in which you are looking for 
peaks

Requirements 
============

Macs2 
IDR 


Usage
=====

See :ref:`PipelineSettingUp` and :ref:`PipelineRunning` on general
information how to use CGAT pipelines.

Configuration
-------------

What is this section?? 

IDR 

Input
-----

Sample_bam = bam file you want to call peaks on 

Input_bam = control file used as background reference in peakcallign 
(e.g. input file for ChIP-seq) 

Pipeline.ini = File containing paramaters and options for
running the pipeline

Desgin.tsv = Design file based on design file for R package Diff Bind 
Has the following collumns: 


Pipeline output
===============

The aim of this pipeline is to output a list of peaks that
can be used for further downstream analysis. 

The pipeline generates several new directories containing 
output files - these can roughly be grouped into XXX main 
stages of the pipeline 

1) Bam file preparation 
   --------------------
   - filtered_bams.dir: 
   Contains bams files (and thier indexes) 
   that have been filtered according to pipeline.ini and a number
   of log files relating to the number of reads that have been 
   filtered out for each reason. Also contains file with the 
   fragment length (insert size) distribution for paired-end 
   samples. 
   
   
   IDR.dir:
   Directory conatining the output files from IDR analysis 
   These include the lists of reproducible peaks and stats and 
   QC tables summarising the output of the IDR analysis
   
   IDR_inputs.dir 
   This directory contains the files that are 

IDR_inputs.dir

macs2.dir/

peakcalling_bams.dir/

peaks_for_IDR.dir/

pooled_bams.dir/
Peaksets:
	
Conservative Peakset = Only obtained if IDR analysis run
IDR analysis 
This analysis does a comparision on a pair of peak files to 
	<commit_msg>update documentatiion<commit_after>"""
pipeline_peakcalling.py - Produce Peaklist from Bam Files 
===================================================

:Author: <NAME> & <NAME>
:Release: $Id$
:Date: |today|
:Tags: Python


Methods
=======

Pipeline Usage
=============

Takes Bam files you want to call peaks in (e.g. ChIP-Seq or ATAC-Seq samples) 
and their appropriate 'input' controls and produces peak lists in bed files
to takeforward for downstream analysis. 

Also runs ChIPQC R package for QC statistics

	Optional functions: 
	-------------------
	- Filter Bam files to remove:
			- Duplicates
			- Secondary alignments
			- Unpaired reads for paired-end files
			- Reads overlapping 'blacklisted' regions 
			- Mapping quality (MAPQ) score
	- Pool input files for peakcalling to give better peakcalling
	  when inputs have poor coverage or lack of sequening depth 
	- Perform Irreproducible Discovery Rate (IDR) analysis (described 
	  further below) to get a consensus list of 
	  'highly reproducible peaks' and assess replicate quaility 


NOTE: WARNINGS!!!! 

1. IDR analysis may not be approprate for all type of peak file - It works 
best with transcription factor CHIPs or methodologies producing 'narrow peaks'
or peaks with well defined boundaries. 

'BroadPeak' IDR (e.g. for widespread histone marks such as H3K27ac)
might not work becuase peak boundary's are harder
to define and thus may not be so reproducible between replicates 

2. Always check your output from this pipeline in a genome browser to check 
peaks are being called suffiently!

3. This pipeline references ChIP bams throughout in the code -this 
referencces the immunoprecipitated (IP) sample from a ChIP experiment 
(i.e. the file you want to find peaks in), Input bams refer to the 
bams of the input control samples that are used for background 
normalisation in peak calling. Although we refer to ChIP bams 
this is only nomenclature and you could just as easily use
an ATAC-Seq bam file or other bam file in which you are looking for 
peaks

Requirements 
============

Macs2 
IDR 


Usage
=====

See :ref:`PipelineSettingUp` and :ref:`PipelineRunning` on general
information how to use CGAT pipelines.

Configuration
-------------

What is this section?? 

IDR 

Input
-----

Sample_bam = bam file you want to call peaks on 

Input_bam = control file used as background reference in peakcalling 
(e.g. input file for ChIP-seq) 

Pipeline.ini = File containing paramaters and options for
running the pipeline

Desgin.tsv = Design file based on design file for R package Diff Bind 
Has the following collumns: 


Pipeline output
===============

The aim of this pipeline is to output a list of peaks that
can be used for further downstream analysis. 

The pipeline generates several new directories containing 
output files - these can roughly be grouped into XXX main 
stages of the pipeline 

1) Bam file preparation 
   --------------------
   - filtered_bams.dir: 
   Contains bams files (and thier indexes) 
   that have been filtered according to pipeline.ini and a number
   of log files relating to the number of reads that have been 
   filtered out for each reason. Also contains file with the 
   frequency of fragment lengths (the 3' end to 5'end of sequenced) for paired-end 
   samples. 
   
   
   IDR.dir:
   Directory conatining the output files from IDR analysis 
   These include the lists of reproducible peaks and stats and 
   QC tables summarising the output of the IDR analysis
   
   IDR_inputs.dir 
   This directory contains the files that are 

IDR_inputs.dir

macs2.dir/

peakcalling_bams.dir/

peaks_for_IDR.dir/

pooled_bams.dir/
Peaksets:
	
Conservative Peakset = Only obtained if IDR analysis run
IDR analysis 
This analysis does a comparision on a pair of peak files to 
	
{'fraction_non_alphanumeric': 0.07819810185804037, 'fraction_numerical': 0.0022724234727977545, 'mean_word_length': 3.5733496332518335, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4507072', 'n_tokens_mistral': 2194, 'n_tokens_neox': 1969, 'n_words': 1021}