<commit_before>unicore.distribute
==================

Distribution tools for Universal Core content.

.. image:: https://travis-ci.org/universalcore/unicore.distribute.svg?branch=develop
    :target: https://travis-ci.org/universalcore/unicore.distribute
    :alt: Continuous Integration

.. image:: https://coveralls.io/repos/universalcore/unicore.distribute/badge.png?branch=develop
    :target: https://coveralls.io/r/universalcore/unicore.distribute?branch=develop
    :alt: Code Coverage

.. image:: https://readthedocs.org/projects/unicoredistribute/badge/?version=latest
    :target: https://unicoredistribute.readthedocs.org
    :alt: unicore.distribute Documentation

.. image:: https://pypip.in/version/unicore.distribute/badge.svg
    :target: https://pypi.python.org/pypi/unicore.distribute
    :alt: Pypi Package

.. image:: docs/unicore.distribute.gif<commit_msg>update README with docker info<commit_after>unicore.distribute
==================

Distribution tools for Universal Core content.

.. image:: https://travis-ci.org/universalcore/unicore.distribute.svg?branch=develop
    :target: https://travis-ci.org/universalcore/unicore.distribute
    :alt: Continuous Integration

.. image:: https://coveralls.io/repos/universalcore/unicore.distribute/badge.png?branch=develop
    :target: https://coveralls.io/r/universalcore/unicore.distribute?branch=develop
    :alt: Code Coverage

.. image:: https://readthedocs.org/projects/unicoredistribute/badge/?version=latest
    :target: https://unicoredistribute.readthedocs.org
    :alt: unicore.distribute Documentation

.. image:: https://pypip.in/version/unicore.distribute/badge.svg
    :target: https://pypi.python.org/pypi/unicore.distribute
    :alt: Pypi Package

Run with Docker
===============

::

    $ docker run -dP sdehaan/unicore.distribute:latest
    1cbc5ea16c08f9787c8d1546f21c489b27bab1c2fb63d7bb97c59565d1d4a2fa
    $ docker port 1cbc5ea16c08
    6543/tcp -> 0.0.0.0:49166
    $ curl 0.0.0.0:49166/repos.json
    []
    $

.. image:: docs/unicore.distribute.gif

{'fraction_non_alphanumeric': 0.15264187866927592, 'fraction_numerical': 0.031800391389432484, 'mean_word_length': 4.395778364116095, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 4, 'https://': 16, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1712235', 'n_tokens_mistral': 745, 'n_tokens_neox': 705, 'n_words': 92}