<commit_before>Simulating reads
================

For each particular combination of sequencing parameters - sequencing depth, read length, single- or paired-end reads, and lack or presence of errors and bias - reads are simulated by running the ``run_simulation.sh`` script in the relevant directory that has been created by the ``piquant.py`` command ``prepare_read_dirs``.

Running ``run_simulation.sh`` results in the following main steps being executed:

Create expression profile
^^^^^^^^^^^^^^^^^^^^^^^^^

FluxSimulator [FluxSimulator]_ is used to create an expression profile (a ``.pro`` file) for the supplied set of transcripts. This profile defines the the set of expressed transcripts, and the relative abundances of those transcripts, from which reads will subsequently be simulated. 

For more information on the model and algorithm used by FluxSimulator to create expression profiles, see `here <http://sammeth.net/confluence/display/SIM/4.1.1+-+Gene+Expression+Profile>`_.

Calculate required number of reads
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Given a particular read length and (approximate) desired sequencing depth, a certain number of reads will need to be simulated. This number is calculated by the support script ``calculate_reads_for_depth.py`` (see :ref:`calculate-reads-for-depth` for more details) and the FluxSimulator simulation parameters file  ``flux_simulator_simulation.par`` updated accordingly.

Simulate reads
^^^^^^^^^^^^^^

Next, FluxSimulator is used to simulate the required number of reads for the desired sequencing depth, according to the previously created transcript expression profile.

Note that:

* Reads are not simulated from the poly-A tails of transcripts (controlled by the FluxSimulator parameters ``POLYA_SHAPE`` and ``POLYA_SCALE``), as the multi-mapping of such reads was found to cause problems for certain quantification tools (for more details on FluxSimulator's transcript modifications, see `here <http://sammeth.net/confluence/display/SIM/4.1.2+-+Transcript+Modifications>`_).
* If sequencing errors have been specified, such errors are simulated with whichever of FluxSimulator's 35bp or 76bp error models is closer in length the the reads being produced. FluxSimulator then scales the error model appropriately (for more details on FluxSimulator's error models, see `here <http://sammeth.net/confluence/display/SIM/4.5.4+-+Error+Models>`_).
* PCR amplification of fragments, controlled by the FluxSimulator parameter ``PCR_DISTRIBUTION`` is not enabled (for more details on FluxSimulator's simulation of PCR, see `here <http://sammeth.net/confluence/display/SIM/4.4.2+-+PCR+Amplification>`_). 

Shuffle reads
^^^^^^^^^^^^^
<commit_msg>Simulating reads may take some time.<commit_after>Simulating reads
================

For each particular combination of sequencing parameters - sequencing depth, read length, single- or paired-end reads, and lack or presence of errors and bias - reads are simulated by running the ``run_simulation.sh`` script in the relevant directory that has been created by the ``piquant.py`` command ``prepare_read_dirs``.

Running ``run_simulation.sh`` results in the following main steps being executed:

Create expression profile
^^^^^^^^^^^^^^^^^^^^^^^^^

FluxSimulator [FluxSimulator]_ is used to create an expression profile (a ``.pro`` file) for the supplied set of transcripts. This profile defines the the set of expressed transcripts, and the relative abundances of those transcripts, from which reads will subsequently be simulated. 

For more information on the model and algorithm used by FluxSimulator to create expression profiles, see `here <http://sammeth.net/confluence/display/SIM/4.1.1+-+Gene+Expression+Profile>`_.

Calculate required number of reads
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Given a particular read length and (approximate) desired sequencing depth, a certain number of reads will need to be simulated. This number is calculated by the support script ``calculate_reads_for_depth.py`` (see :ref:`calculate-reads-for-depth` for more details) and the FluxSimulator simulation parameters file  ``flux_simulator_simulation.par`` updated accordingly.

Simulate reads
^^^^^^^^^^^^^^

Next, FluxSimulator is used to simulate the required number of reads for the desired sequencing depth, according to the previously created transcript expression profile. Note that depending on the number of reads being simulated, this step can take some time.

Note that:

* Reads are not simulated from the poly-A tails of transcripts (controlled by the FluxSimulator parameters ``POLYA_SHAPE`` and ``POLYA_SCALE``), as the multi-mapping of such reads was found to cause problems for certain quantification tools (for more details on FluxSimulator's transcript modifications, see `here <http://sammeth.net/confluence/display/SIM/4.1.2+-+Transcript+Modifications>`_).
* If sequencing errors have been specified, such errors are simulated with whichever of FluxSimulator's 35bp or 76bp error models is closer in length the the reads being produced. FluxSimulator then scales the error model appropriately (for more details on FluxSimulator's error models, see `here <http://sammeth.net/confluence/display/SIM/4.5.4+-+Error+Models>`_).
* PCR amplification of fragments, controlled by the FluxSimulator parameter ``PCR_DISTRIBUTION`` is not enabled (for more details on FluxSimulator's simulation of PCR, see `here <http://sammeth.net/confluence/display/SIM/4.4.2+-+PCR+Amplification>`_). 

Shuffle reads
^^^^^^^^^^^^^

{'fraction_non_alphanumeric': 0.10498640072529465, 'fraction_numerical': 0.005802357207615594, 'mean_word_length': 5.7104622871046224, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3262848', 'n_tokens_mistral': 1496, 'n_tokens_neox': 1364, 'n_words': 637}