<commit_before>pymining - A collection of data mining algorithms in Python
===========================================================

:Authors:
  <NAME>
:Version: 0.1

pymining is a small collection of data mining algorithms implemented in Python.
I did not design any of the algorithms, but I use them in my own research so I
thought other developers might be interested to use them as well.

I started this small project because I could not find data mining algorithms
that were easily accessible in Python. Moreover, the libraries I am aware of
often include old algorithms that have been surpassed by newer ones.


Requirements
------------

pymining has been tested with Python 2.7 and 3.2.


Installation
------------

::

    pip install -e git://github.com/bartdag/pymining#egg=pymining


Usage
-----

**Frequent Item Set Mining**

::

    >>> from pymining import itemmining
    >>> transactions = (('a', 'b', 'c'), ('b'), ('a'), ('a', 'c', 'd'), ('b', 'c'), ('b', 'c'))
    >>> relim_input = itemmining.get_relim_input(transactions)
    >>> report = itemmining.relim(relim_input, min_support=2)
    >>> report
    set([(('c',), 4), (('b',), 4), (('a', 'c'), 2), (('a',), 3), (('c', 'b'), 3)])

    >>> # Test performance of multiple algorithms
    >>> itemmining.test_perf()
    Random transactions generated

    Done round 0
    Done round 1
    ...


Status of the project
---------------------

Three algorithms are currently implemented to find frequent item sets. Relim is
the recommended algorithm as it outperforms the two others (SaM and FP-growth)
in all of my benchmarks. This is probably due to my lazy implementation of
FP-growth.


Todo
----<commit_msg>added a warning about a bug<commit_after>pymining - A collection of data mining algorithms in Python
===========================================================

:Authors:
  <NAME>
:Version: 0.1

pymining is a small collection of data mining algorithms implemented in Python.
I did not design any of the algorithms, but I use them in my own research so I
thought other developers might be interested to use them as well.

I started this small project because I could not find data mining algorithms
that were easily accessible in Python. Moreover, the libraries I am aware of
often include old algorithms that have been surpassed by newer ones.


Requirements
------------

pymining has been tested with Python 2.7 and 3.2.


Installation
------------

::

    pip install -e git://github.com/bartdag/pymining#egg=pymining


Usage
-----

**Frequent Item Set Mining**

::

    >>> from pymining import itemmining
    >>> transactions = (('a', 'b', 'c'), ('b'), ('a'), ('a', 'c', 'd'), ('b', 'c'), ('b', 'c'))
    >>> relim_input = itemmining.get_relim_input(transactions)
    >>> report = itemmining.relim(relim_input, min_support=2)
    >>> report
    set([(('c',), 4), (('b',), 4), (('a', 'c'), 2), (('a',), 3), (('c', 'b'), 3)])

    >>> # Test performance of multiple algorithms
    >>> itemmining.test_perf()
    Random transactions generated

    Done round 0
    Done round 1
    ...


Status of the project
---------------------

Three algorithms are currently implemented to find frequent item sets. Relim is
the recommended algorithm as it outperforms the two others (SaM and FP-growth)
in all of my benchmarks. This is probably due to my lazy implementation of
FP-growth.

The pruning option in FP-growth does not work and should be turned to False for
now.


Todo
----
{'fraction_non_alphanumeric': 0.17235396182764604, 'fraction_numerical': 0.008097165991902834, 'mean_word_length': 3.3951715374841167, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '162118', 'n_tokens_mistral': 1031, 'n_tokens_neox': 938, 'n_words': 431}