<commit_before>Installation
------------

Automated
=========

We provide an automated script that installs third party analysis tools,
required genome data and python library dependencies for running human variant
and RNA-seq analysis, bundled into an isolated directory or virtual environment::

     wget https://raw.github.com/chapmanb/bcbio-nextgen/master/scripts/bcbio_nextgen_install.py
     python bcbio_nextgen_install.py /usr/local/share/bcbio --tooldir=/usr/local \
       --genomes GRCh37 --aligners bwa --aligners bowtie2

bcbio should install cleanly on most Linux systems. For Mac OSX, we suggest
trying `bcbio-vm <https://github.com/chapmanb/bcbio-nextgen-vm>`_ which runs
bcbio on :ref:`docs-cloud` or isolates all the third party tools inside a
Docker container.

It places genomes, indexes and associated data files in
``/usr/local/share/bcbio-nextgen`` and tools in ``/usr/local``. You should edit
the pre-created system configuration file in
``/usr/local/share/bcbio-nextgen/galaxy/bcbio_system.yaml``
to match your local system or cluster configuration.

The installation is highly customizable, and you can install
additional software and data later using ``bcbio_nextgen.py upgrade``.
Run ``python bcbio_nextgen_install.py`` with no arguments to see options
for configuring the installation process. Some useful arguments are:

- ``--sudo`` Enable installation in privileged directories and allow the
  installer to update system packages.
- ``--isolate`` Avoid updating the user's ``~/.bashrc`` if installing in a
  non-standard PATH. This facilitates creation of isolated modules
  without disrupting the user's environmental setup.
- ``--nodata`` Do not install genome data.

To bootstrap installation, the machine will need to have some basic
requirements:

- Python 2.7, Python 3.x or Python 2.6 plus the argparse dependency
  with the development libraries installed (the python-dev or python-devel
  packages).
- Compilers: Recent versions of gcc, g++ and gfortran. gcc 4.8.x
  is well tested, although other versions should work.
- The git version control system (http://git-scm.com/).
- wget for file retrieval (https://www.gnu.org/software/wget/)
- unzip
- zlib (with development libraries)

If you're not using the ``--sudo`` option, please see :ref:`isolated-install`
for additional system requirements needed to bootstrap the full system on
minimal machines. The
`bcbio-nextgen Dockerfile <https://github.com/chapmanb/bcbio-nextgen/blob/master/Dockerfile#L5>`_
contains bootstrap package information to install on bare Ubuntu systems.

The automated installer creates a fully integrated environment that
allows simultaneous updates of the framework, third party tools and
biological data. This offers the advantage over manual installation of
being able to manage and evolve a consistent analysis environment as
algorithms continue to evolve and improve. The installer is flexible
enough to handle both system integrations into standard directories
like /usr/local, as well as custom isolated installations in non-root
directories. The :ref:`upgrade-install` section has additional
documentation on including additional genome data, and the section on
:ref:`toolplus-install` describes how to add commercially restricted software
like GATK.

.. _isolated-install:

Isolated installations
======================

To install bcbio-nextgen in an isolated non-root environment::

    python bcbio_nextgen_install.py /path_to_bcbio --tooldir=/path_to_bcbio --isolate \
       --genomes GRCh37 --aligners bwa --aligners bowtie2

This requires the following additional system requirements to be in place:

- Java 1.7
- Ruby (including libraries and irb. On recent CentOS and other rpm systems
  these are separate packages:
  ``ruby-libs`` and ``ruby-irb``)
- Perl with development libraries (On Ubuntu or other deb systems:
  ``perl, libperl-devel`` On RedHat or other rpm systems:
  ``perl perl-devel perl-core``)
- bzip2 (with development libraries)
- curl and SSL (with development libraries; On Ubuntu: ``libssl-dev libcurl4-openssl-dev``, On
  RedHat: ``openssl-devel libcurl-devel``). Building R and git on older systems requires a relatively
  recent version of curl (newer than v7.28).
- XML development libraries (On Ubuntu: ``libxml2-dev``, on RedHat: ``libxml2-devel``)
- curses with development libraries (On Ubuntu: ``libncurses5-dev``, on RedHat ``ncurses-devel``)
<commit_msg>Package name typo fix<commit_after>Installation
------------

Automated
=========

We provide an automated script that installs third party analysis tools,
required genome data and python library dependencies for running human variant
and RNA-seq analysis, bundled into an isolated directory or virtual environment::

     wget https://raw.github.com/chapmanb/bcbio-nextgen/master/scripts/bcbio_nextgen_install.py
     python bcbio_nextgen_install.py /usr/local/share/bcbio --tooldir=/usr/local \
       --genomes GRCh37 --aligners bwa --aligners bowtie2

bcbio should install cleanly on most Linux systems. For Mac OSX, we suggest
trying `bcbio-vm <https://github.com/chapmanb/bcbio-nextgen-vm>`_ which runs
bcbio on :ref:`docs-cloud` or isolates all the third party tools inside a
Docker container.

It places genomes, indexes and associated data files in
``/usr/local/share/bcbio-nextgen`` and tools in ``/usr/local``. You should edit
the pre-created system configuration file in
``/usr/local/share/bcbio-nextgen/galaxy/bcbio_system.yaml``
to match your local system or cluster configuration.

The installation is highly customizable, and you can install
additional software and data later using ``bcbio_nextgen.py upgrade``.
Run ``python bcbio_nextgen_install.py`` with no arguments to see options
for configuring the installation process. Some useful arguments are:

- ``--sudo`` Enable installation in privileged directories and allow the
  installer to update system packages.
- ``--isolate`` Avoid updating the user's ``~/.bashrc`` if installing in a
  non-standard PATH. This facilitates creation of isolated modules
  without disrupting the user's environmental setup.
- ``--nodata`` Do not install genome data.

To bootstrap installation, the machine will need to have some basic
requirements:

- Python 2.7, Python 3.x or Python 2.6 plus the argparse dependency
  with the development libraries installed (the python-dev or python-devel
  packages).
- Compilers: Recent versions of gcc, g++ and gfortran. gcc 4.8.x
  is well tested, although other versions should work.
- The git version control system (http://git-scm.com/).
- wget for file retrieval (https://www.gnu.org/software/wget/)
- unzip
- zlib (with development libraries)

If you're not using the ``--sudo`` option, please see :ref:`isolated-install`
for additional system requirements needed to bootstrap the full system on
minimal machines. The
`bcbio-nextgen Dockerfile <https://github.com/chapmanb/bcbio-nextgen/blob/master/Dockerfile#L5>`_
contains bootstrap package information to install on bare Ubuntu systems.

The automated installer creates a fully integrated environment that
allows simultaneous updates of the framework, third party tools and
biological data. This offers the advantage over manual installation of
being able to manage and evolve a consistent analysis environment as
algorithms continue to evolve and improve. The installer is flexible
enough to handle both system integrations into standard directories
like /usr/local, as well as custom isolated installations in non-root
directories. The :ref:`upgrade-install` section has additional
documentation on including additional genome data, and the section on
:ref:`toolplus-install` describes how to add commercially restricted software
like GATK.

.. _isolated-install:

Isolated installations
======================

To install bcbio-nextgen in an isolated non-root environment::

    python bcbio_nextgen_install.py /path_to_bcbio --tooldir=/path_to_bcbio --isolate \
       --genomes GRCh37 --aligners bwa --aligners bowtie2

This requires the following additional system requirements to be in place:

- Java 1.7
- Ruby (including libraries and irb. On recent CentOS and other rpm systems
  these are separate packages:
  ``ruby-libs`` and ``ruby-irb``)
- Perl with development libraries (On Ubuntu or other deb systems:
  ``perl, libperl-dev`` On RedHat or other rpm systems:
  ``perl perl-devel perl-core``)
- bzip2 (with development libraries)
- curl and SSL (with development libraries; On Ubuntu: ``libssl-dev libcurl4-openssl-dev``, On
  RedHat: ``openssl-devel libcurl-devel``). Building R and git on older systems requires a relatively
  recent version of curl (newer than v7.28).
- XML development libraries (On Ubuntu: ``libxml2-dev``, on RedHat: ``libxml2-devel``)
- curses with development libraries (On Ubuntu: ``libncurses5-dev``, on RedHat ``ncurses-devel``)

{'fraction_non_alphanumeric': 0.08949152542372882, 'fraction_numerical': 0.005423728813559322, 'mean_word_length': 4.545739348370927, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1184525', 'n_tokens_mistral': 2572, 'n_tokens_neox': 2370, 'n_words': 1039}