<commit_before>compound-word-splitter
======================

.. image:: https://travis-ci.org/TimKam/compound-word-splitter.svg?branch=master
    :target: https://travis-ci.org/TimKam/compound-word-splitter

Splits words that are not recognized by pyenchant (spell checker) into largest possible compounds.

Installation
------------

::

    pip install compound-word-splitter

Usage
-----

.. code:: python

    import splitter

    splitter.split('artfactory')

returns

.. code:: python

    ['art', 'factory']

If the word cannot be split into compounds pyenchant recognizes as words, the splitter returns an empty string.

<commit_msg>Add German example<commit_after>compound-word-splitter
======================

.. image:: https://travis-ci.org/TimKam/compound-word-splitter.svg?branch=master
    :target: https://travis-ci.org/TimKam/compound-word-splitter

Splits words that are not recognized by pyenchant (spell checker) into largest possible compounds.

Installation
------------

::

    pip install compound-word-splitter

Usage
-----

.. code:: python

    import splitter

    splitter.split('artfactory')

returns

.. code:: python

    ['art', 'factory']
    
.. code:: python

    split('Glossarelement', 'de_de')
   
returns

.. code:: python

    ['Glossar', 'Element']

If the word cannot be split into compounds pyenchant recognizes as words, the splitter returns an empty string.


{'fraction_non_alphanumeric': 0.15778251599147122, 'fraction_numerical': 0.0, 'mean_word_length': 3.4276729559748427, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '280964', 'n_tokens_mistral': 436, 'n_tokens_neox': 414, 'n_words': 117}