<commit_before>Sysadmin
=========

Installation
-------------

System dependencies
^^^^^^^^^^^^^^^^^^^^
.. note::
        In this moment we do *not* support elasticsearch v2.
        There are plans to do it shortly!

Debian wheezy / Debian jessie / Ubuntu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. highlight:: bash

Download and install the Public Signing Key for elasticsearch repo::

    wget -qO - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

Add elasticsearch repos in /etc/apt/sources.list.d/elasticsearch.list::

    echo "deb http://packages.elasticsearch.org/elasticsearch/1.7/debian stable main" | sudo tee /etc/apt/sources.list.d/elasticsearch.list

Install requirements::
    
    sudo apt-get update && sudo apt-get install python2.7 gcc python2.7-dev python-virtualenv openjdk-7-jre-headless elasticsearch

.. note::
    
    if you have problem installing elasticsearch try to follow the `official installation guide`_

.. _official installation guide: http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html

Arch
~~~~~

Install all necessary packages::

    sudo pacman -Sy python2 python2-virtualenv 

And take care to install elasticsearch<2.x::

    wget https://archive.archlinux.org/packages/e/elasticsearch/elasticsearch-1.7.3-1-x86_64.pkg.tar.xz
    wget https://archive.archlinux.org/packages/e/elasticsearch/elasticsearch-1.7.3-1-x86_64.pkg.tar.xz.sig
    sudo pacman-key --verify elasticsearch-1.7.3-1-x86_64.pkg.tar.xz elasticsearch-1.7.3-1-x86_64.pkg.tar.xz.sig
    sudo pacman -U elasticsearch-1.7.3.1-x86_64.pkg.tar.xz
    echo "IgnorePkg elasticsearch" | sudo tee -a /etc/pacman.conf

Python dependencies
^^^^^^^^^^^^^^^^^^^^

Create a virtual env::

    virtualenv -p /usr/bin/python2 ve

Install libreant and all python dependencies::
    
    ./ve/bin/pip install libreant

Execution
----------

Start elsticsearch
^^^^^^^^^^^^^^^^^^^

Debian wheezy / Ubuntu
~~~~~~~~~~~~~~~~~~~~~~

Start elasticsearch service::

    sudo service elasticsearch start

.. note::<commit_msg>Installation steps upgraded to elasticsearch2<commit_after>Sysadmin
=========

.. _sys-Installation:

Installation
-------------

System dependencies
^^^^^^^^^^^^^^^^^^^^

Debian wheezy / Debian jessie / Ubuntu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. highlight:: bash

Download and install the Public Signing Key for elasticsearch repo::

    wget -qO - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

Add elasticsearch repos in /etc/apt/sources.list.d/elasticsearch.list::

    echo "deb http://packages.elasticsearch.org/elasticsearch/2.x/debian stable main" | sudo tee /etc/apt/sources.list.d/elasticsearch.list

Install requirements::
    
    sudo apt-get update && sudo apt-get install python2.7 gcc python2.7-dev python-virtualenv openjdk-7-jre-headless elasticsearch

.. note::
    
    if you have problem installing elasticsearch try to follow the `official installation guide`_

.. _official installation guide: http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html

Arch
~~~~~

Install all necessary packages::

    sudo pacman -Sy python2 python2-virtualenv elasticsearch

Python dependencies
^^^^^^^^^^^^^^^^^^^^

Create a virtual env::

    virtualenv -p /usr/bin/python2 ve

Install libreant and all python dependencies::
    
    ./ve/bin/pip install libreant

Execution
----------

Start elsticsearch
^^^^^^^^^^^^^^^^^^^

Debian wheezy / Ubuntu
~~~~~~~~~~~~~~~~~~~~~~

Start elasticsearch service::

    sudo service elasticsearch start

.. note::
{'fraction_non_alphanumeric': 0.17175360710321866, 'fraction_numerical': 0.017203107658157604, 'mean_word_length': 3.657622739018088, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1168869', 'n_tokens_mistral': 1138, 'n_tokens_neox': 1059, 'n_words': 278}