<commit_before>django-pq
==========

A task queue based on the elegant [RQ](http://python-rq.org) but with a django postgresql backend.

RQ sets a low barrier for entry, and django-pq takes it lower for sites that can’t or don’t want to use Redis in their stack, and are happy to trade off performance for the transactional integrity of Postgres. As additional throughput is required you should be able to switch out or mix django-pq with the more performant Redis based RQ with only trival code changes.

Django-pq is currently considered alpha quality, and is probably not suitable for production.

Installation
--------------

Currently only available through github:

    $ pip install https://github.com/bretth/django-pq/zipball/master

Getting started
----------------

If you have used RQ then you’ll know django-pq but lets start with the RQ example.

    import requests

    def count_words_at_url(url):
        resp = requests.get(url)
        return len(resp.text.split())

Create the queue.

    from pq import Queue
    q = Queue()

Enqueue the function.

    q.enqueue(count_words_at_url, 'http://python-rq.org')


Consume your queue with a worker.

    $ pqworker —burst
     *** Listening for work on default
     Got count_words_at_url('http://nvie.com') from default
      Job result = 818
     *** Listening for work on default

That’s enough to get started. Lets look in some more detail.

Queues
---------

Since django-pq is uses django models we have one piece of syntactic sugar to maintain compatibility with RQ.

    from pq import Queue
    # create a default queue called ‘default’
    queue = Queue()

Is syntactic sugar for:

    from  pq.queue import Queue
    queue = Queue.create()

Lets create a queue called ‘slow’ with a 10 minute timeout on jobs (up from the default 3 minutes) that uses a different database from the django ‘default’ db and runs synchronously if DEBUG:

    from django.conf import settings
    slowq = Queue(‘slow’,  600, connection=‘somedb’,  async=settings.DEBUG)<commit_msg>Less docs<commit_after>django-pq
==========

A task queue based on the elegant [RQ](http://python-rq.org) but with a django postgresql backend.

RQ sets a low barrier for entry, and django-pq takes it lower for sites that can’t or don’t want to use Redis in their stack, and are happy to trade off performance for the transactional integrity of Postgres. As additional throughput is required you should be able to switch out or mix django-pq with the more performant Redis based RQ with only trival code changes.

Django-pq is currently considered alpha quality, and is probably not suitable for production.

Installation
--------------

Currently only available through github:

    $ pip install https://github.com/bretth/django-pq/zipball/master

Getting started
----------------

If you have used RQ then you’ll know django-pq but lets start with the RQ example.

    import requests

    def count_words_at_url(url):
        resp = requests.get(url)
        return len(resp.text.split())

Create the queue.

    from pq import Queue
    q = Queue()

Enqueue the function.

    q.enqueue(count_words_at_url, 'http://python-rq.org')


Consume your queue with a worker.

    $ pqworker —burst
     *** Listening for work on default
     Got count_words_at_url('http://nvie.com') from default
      Job result = 818
     *** Listening for work on default


Queues
---------

Since django-pq is uses django models we have one piece of syntactic sugar to maintain compatibility with RQ.

    from pq import Queue
    # create a default queue called ‘default’
    queue = Queue()

Is syntactic sugar for:

    from  pq.queue import Queue
    queue = Queue.create()

Lets create a queue called ‘slow’ with a 10 minute timeout on jobs (up from the default 3 minutes) that uses a different database from the django ‘default’ db and runs synchronously if DEBUG:

    from django.conf import settings
    slowq = Queue(‘slow’,  600, connection=‘somedb’,  async=settings.DEBUG)
{'fraction_non_alphanumeric': 0.08568573569822634, 'fraction_numerical': 0.0044966275293529855, 'mean_word_length': 3.219178082191781, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1388631', 'n_tokens_mistral': 1162, 'n_tokens_neox': 1086, 'n_words': 513}