<commit_before># BigData Republic Analytics (python)
Our analytics library to quickly get our data scientists up to speed, on the python platform

## Recommended setup
If you need an environment including this module, the recommended way to go is:
> git clone <EMAIL>:BigDataRepublic/bdr-analytics-py.git

> cd bdr-analytics-py

> pyenv install 2.7.11

You can use the bdranalytics env, or your project specific environment:
> pyenv virtualenv 2.7.11 bdranalytics

> pyenv activate bdranalytics

> pip install --upgrade pip

This is the line which will install all requirements and the module itself
> pip install -r requirements.txt

## Setup details and alternative setups
This package has dependencies and requirements:
  * dependencies will be always be installed through a pip install of the module / setup.py
  * requirements are required before installing the dependencies.
    The most important one is sklearn version 1.8 (currently on scikit-learns git repo master)

To install the package without checking it out (to use this module):
  * First check `requirements.txt` and install the correct requirements (sklearn), including their dependencies.
  * Then install the bdr-analytics-py including dependencies:
    > pip install -e git+ssh://git@github.com/BigDataRepublic/bdr-analytics.git#egg=bdranalytics-0.1

To install the package including both its requiremnets and dependencies:
  * First checkout from github
    > git clone git@github.com:BigDataRepublic/bdr-analytics-py.git

    > cd bdr-analytics-py
  * Then install bdr-analytics-py including requirements and dependencies:
    > pip install -r requirements.txt


To build the package from a local checkout (when developing on this module):
> python setup.py sdist<commit_msg>small layout improvements on readme<commit_after># BigData Republic Analytics (python)
Our analytics library to quickly get our data scientists up to speed, on the python platform

## Recommended setup
If you need an environment including this module, the recommended way to go is:
> git clone <EMAIL>:BigDataRepublic/bdr-analytics-py.git

> cd bdr-analytics-py

> pyenv install 2.7.11

You can use the bdranalytics env, or your project specific environment:
> pyenv virtualenv 2.7.11 bdranalytics

> pyenv activate bdranalytics

> pip install --upgrade pip

This is the line which will install all requirements, dependencies and the module itself
> pip install -r requirements.txt

## Setup details and alternative setups
This package has dependencies and requirements:
  * dependencies will be always be installed through a pip install of the module / setup.py
  * requirements are required before installing the dependencies.
    The most important one is sklearn version 1.8 (currently on scikit-learns git repo master)

#### Without git checkout
To install the package without checking it out (to use this module):
  * First check `requirements.txt` and install the correct requirements (sklearn), including their dependencies.
  * Then install the bdr-analytics-py including dependencies:

    > pip install -e git+ssh://git@github.com/BigDataRepublic/bdr-analytics.git#egg=bdranalytics-0.1

#### With git checkout
To install the package including both its requiremets and dependencies:
  * First checkout from github

    > git clone git@github.com:BigDataRepublic/bdr-analytics-py.git

    > cd bdr-analytics-py
  * Then install bdr-analytics-py including requirements and dependencies:

    > pip install -r requirements.txt


To create a dist from a local checkout (when developing on this module):
> python setup.py sdist
{'fraction_non_alphanumeric': 0.06115610164758448, 'fraction_numerical': 0.006702038536721586, 'mean_word_length': 4.4520547945205475, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2419199', 'n_tokens_mistral': 969, 'n_tokens_neox': 929, 'n_words': 430}