<commit_before>CountMinSketch
==============

This is a minimalistic
`Count-min Sketch <http://en.wikipedia.org/wiki/Count%E2%80%93min_sketch>`_
for Python, featuring some cool things like:
 - Being able to count anything that is hash-able by python (numbers, strings, tuples, inmutables, duck-typeds, etc.).
 - Tests
 - No dependencies
 - No foreign languages, just 100% python.


Usage
=====

.. code-block:: python

    from countminsketch import CountMinSketch
    sketch = CountMinSketch(1000, 10)  # 10 hash functions, 1000 places per hash function
    sketch.update("oh yeah")
    sketch.update(tuple())
    sketch.update(1, value=123)
    print sketch["oh yeah"]       # prints 1
    print sketch[tuple()]         # prints 1
    print sketch[1]               # prints 123
    print sketch["non-existent"]  # prints 0<commit_msg>More readme rst magic<commit_after>CountMinSketch
==============

This is a minimalistic
`Count-min Sketch <http://en.wikipedia.org/wiki/Count%E2%80%93min_sketch>`_
for Python, featuring some cool things like:
- Being able to count anything that is hash-able by python (numbers, strings, tuples, inmutables, duck-typeds, etc.).
- Tests
- No dependencies
- No foreign languages, just 100% python.


Usage
=====

.. code-block:: python

    from countminsketch import CountMinSketch
    sketch = CountMinSketch(1000, 10)  # 10 hash functions, 1000 places per hash function
    sketch.update("oh yeah")
    sketch.update(tuple())
    sketch.update(1, value=123)
    print sketch["oh yeah"]       # prints 1
    print sketch[tuple()]         # prints 1
    print sketch[1]               # prints 123
    print sketch["non-existent"]  # prints 0
{'fraction_non_alphanumeric': 0.12171837708830549, 'fraction_numerical': 0.03699284009546539, 'mean_word_length': 3.413157894736842, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1583652', 'n_tokens_mistral': 550, 'n_tokens_neox': 493, 'n_words': 162}