<commit_before>PimPamQueues
============

**Lightweight queue interfaces with Redis super powers**

.. image:: https://api.travis-ci.org/jordimarinvalle/pimpamqueues.png
    :target: https://secure.travis-ci.org/jordimarinvalle/pimpamqueues

.. image:: https://pypip.in/version/pimpamqueues/badge.svg?text=version
    :target: https://pypi.python.org/pypi/pimpamqueues/
    :alt: Latest Version

.. image:: https://pypip.in/py_versions/pimpamqueues/badge.svg
    :target: https://pypi.python.org/pypi/pimpamqueues/
    :alt: Supported Python versions


Description
-----------
PimPamQueues provides easy and lightweight Python interfaces to interact with queues on a distributed system.


Requirements
------------
- Python 2.7 or 3.4, `python.org <https://www.python.org/>`_
- A runing Redis server, `redis.io <http://redis.io/>`_
- Redis Python library, `redis-py.readthedocs.org <https://redis-py.readthedocs.org/en/latest/>`_


Features
--------
- Supports at least Python 2.7 and 3.4.
- Provides super-simple queue interfaces for creating different types of queues.
- Designed to be used on distributed systems - also works on non-distributed systems ;).


Queue Interfaces
----------------
- SimpleQueue, just a regular queue.
- BucketQueue, unordered queue of unique elements with a extreamly fast element existance search method.
- SmartQueue, queue which stores queued elements aside the queue for not queueing the same incoming elements again.


Usage
-----

SimpleQueue
~~~~~~~~~~~

.. code:: bash

    >>> from pimpamqueues.simplequeue import SimpleQueue
    >>> queue = SimpleQueue(id_args=['simplequeue'])
    >>> queue.num()
    0
    >>> queue.push('egg')
    1
    >>> queue.push_some(['bacon', 'spam'])
    3
    >>> queue.num()
    3
    >>> queue.pop()
    b'egg'
    >>> queue.is_empty()
    False
    >>> queue.push('spam', to_first=True)
    3
    >>> queue.elements()
    [b'spam', b'bacon', b'spam']
    >>> queue.pop()
    b'spam'
    >>> queue.elements()
    [b'bacon', b'spam']
    ...


BucketQueue
~~~~~~~~~~~

.. code:: bash

    >>> from pimpamqueues.bucketqueue import BucketQueue
    >>> queue = BucketQueue(id_args=['bucketqueue'])
    >>> queue.num()
    0
    >>> queue.push('egg')
    'egg'
    >>> queue.push_some(['bacon', 'spam'])
    [b'bacon', b'spam']
    >>> queue.num()
    3
    >>> queue.pop()
    b'spam'
    >>> queue.is_empty()
    False
    >>> queue.push('spam')
    'spam'
    >>> queue.elements()
    {b'bacon', b'egg', b'spam'}
    >>> queue.pop()
    b'spam'
    >>> queue.elements()
    {b'bacon', b'egg'}
    ...


SmartQueue
~~~~~~~~~~

.. code:: bash

    >>> from pimpamqueues.smartqueue import SmartQueue
    >>> queue = SmartQueue(id_args=['smartqueue'])
    >>> queue.num()
    0
    >>> queue.push('egg')
    'egg'
    >>> queue.push_some(['bacon', 'spam'])
    [b'bacon', b'spam']
    >>> queue.num()
    3
    >>> queue.pop()
    b'egg'
    >>> queue.is_empty()
    False
    >>> queue.push('spam', to_first=True)
    ''
    >>> queue.elements()
    [b'bacon', b'spam']
    >>> queue.pop()
    b'bacon'
    >>> queue.elements()
    [b'spam']
    ...
<commit_msg>readme file, pip install pimpamqueues<commit_after>PimPamQueues
============

**Lightweight queue interfaces with Redis super powers**

.. image:: https://api.travis-ci.org/jordimarinvalle/pimpamqueues.png
    :target: https://secure.travis-ci.org/jordimarinvalle/pimpamqueues

.. image:: https://pypip.in/version/pimpamqueues/badge.svg?text=version
    :target: https://pypi.python.org/pypi/pimpamqueues/
    :alt: Latest Version

.. image:: https://pypip.in/py_versions/pimpamqueues/badge.svg
    :target: https://pypi.python.org/pypi/pimpamqueues/
    :alt: Supported Python versions


Description
-----------
PimPamQueues provides easy and lightweight Python interfaces to interact with queues on a distributed system.


Requirements
------------
- Python 2.7 or 3.4, `python.org <https://www.python.org/>`_
- A runing Redis server, `redis.io <http://redis.io/>`_
- Redis Python library, `redis-py.readthedocs.org <https://redis-py.readthedocs.org/en/latest/>`_


Features
--------
- Supports at least Python 2.7 and 3.4.
- Provides super-simple queue interfaces for creating different types of queues.
- Designed to be used on distributed systems - also works on non-distributed systems ;).


Queue Interfaces
----------------
- SimpleQueue, just a regular queue.
- BucketQueue, unordered queue of unique elements with a extreamly fast element existance search method.
- SmartQueue, queue which stores queued elements aside the queue for not queueing the same incoming elements again.


Instalation
-----------

PIP
~~~

For a pip installation, just ``pip install pimpamqueues``

.. code:: bash

    pip install pimpamqueues



Usage
-----

SimpleQueue
~~~~~~~~~~~

.. code:: bash

    >>> from pimpamqueues.simplequeue import SimpleQueue
    >>> queue = SimpleQueue(id_args=['simplequeue'])
    >>> queue.num()
    0
    >>> queue.push('egg')
    1
    >>> queue.push_some(['bacon', 'spam'])
    3
    >>> queue.num()
    3
    >>> queue.pop()
    b'egg'
    >>> queue.is_empty()
    False
    >>> queue.push('spam', to_first=True)
    3
    >>> queue.elements()
    [b'spam', b'bacon', b'spam']
    >>> queue.pop()
    b'spam'
    >>> queue.elements()
    [b'bacon', b'spam']
    ...


BucketQueue
~~~~~~~~~~~

.. code:: bash

    >>> from pimpamqueues.bucketqueue import BucketQueue
    >>> queue = BucketQueue(id_args=['bucketqueue'])
    >>> queue.num()
    0
    >>> queue.push('egg')
    'egg'
    >>> queue.push_some(['bacon', 'spam'])
    [b'bacon', b'spam']
    >>> queue.num()
    3
    >>> queue.pop()
    b'spam'
    >>> queue.is_empty()
    False
    >>> queue.push('spam')
    'spam'
    >>> queue.elements()
    {b'bacon', b'egg', b'spam'}
    >>> queue.pop()
    b'spam'
    >>> queue.elements()
    {b'bacon', b'egg'}
    ...


SmartQueue
~~~~~~~~~~

.. code:: bash

    >>> from pimpamqueues.smartqueue import SmartQueue
    >>> queue = SmartQueue(id_args=['smartqueue'])
    >>> queue.num()
    0
    >>> queue.push('egg')
    'egg'
    >>> queue.push_some(['bacon', 'spam'])
    [b'bacon', b'spam']
    >>> queue.num()
    3
    >>> queue.pop()
    b'egg'
    >>> queue.is_empty()
    False
    >>> queue.push('spam', to_first=True)
    ''
    >>> queue.elements()
    [b'bacon', b'spam']
    >>> queue.pop()
    b'bacon'
    >>> queue.elements()
    [b'spam']
    ...

{'fraction_non_alphanumeric': 0.19237354085603112, 'fraction_numerical': 0.005291828793774319, 'mean_word_length': 2.836417910447761, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 225, 'https://': 16, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4953159', 'n_tokens_mistral': 2288, 'n_tokens_neox': 2205, 'n_words': 479}