<commit_before>Configuration
-------------
.. py:currentmodule:: django_q

Configuration is handled via the ``Q_CLUSTER`` dictionary in your :file:`settings.py`

.. code:: python

    # settings.py example
    Q_CLUSTER = {
        'name': 'myproject',
        'workers': 8,
        'recycle': 500,
        'timeout': 60,
        'compress': True,
        'save_limit': 250,
        'queue_limit': 500,
        'cpu_affinity': 1,
        'label': 'Django Q',
        'redis': {
            'host': '127.0.0.1',
            'port': 6379,
            'db': 0, }
    }

All configuration settings are optional:

name
~~~~

Used to differentiate between projects using the same Redis server. Defaults to ``'default'``.
This can be useful if you have several projects using the same Redis server.

.. note::
    Tasks are encrypted. When a worker encounters a task it can not decrypt, it will be discarded.

workers
~~~~~~~

The number of workers to use in the cluster. Defaults to CPU count of the current host, but can be set to a custom number.  [#f1]_

recycle
~~~~~~~

The number of tasks a worker will process before recycling . Useful to release memory resources on a regular basis. Defaults to ``500``.

.. _timeout:

timeout
~~~~~~~

The number of seconds a worker is allowed to spend on a task before it's terminated. Defaults to ``None``, meaning it will never time out.
Set this to something that makes sense for your project. Can be overridden for individual tasks.

.. _retry:

retry
~~~~~

The number of seconds a broker will wait for a cluster to finish a task, before it's presented again.
Only works with brokers that support delivery receipts. Defaults to 60 seconds.

compress
~~~~~~~~

Compresses task packages to Redis. Useful for large payloads, but can add overhead when used with many small packages.
Defaults to ``False``

.. _save_limit:

save_limit
~~~~~~~~~~

Limits the amount of successful tasks saved to Django.
 - Set to ``0`` for unlimited.
 - Set to ``-1`` for no success storage at all.
 - Defaults to ``250``
 - Failures are always saved.

.. _sync:

sync
~~~~

When set to ``True`` this configuration option forces all :func:`async` calls to be run with ``sync=True``.
Effectively making everything synchronous. Useful for testing. Defaults to ``False``.

.. _queue_limit:

queue_limit
~~~~~~~~~~~

This does not limit the amount of tasks that can be queued on the broker, but rather how many tasks are kept in memory by a single cluster.
Setting this to a reasonable number, can help balance the workload and the memory overhead of each individual cluster.
Defaults to ``workers**2``.

label
~~~~~

The label used for the Django Admin page. Defaults to ``'Django Q'``

.. _catch_up:

catch_up
~~~~~~~~
The default behavior for schedules that didn't run while a cluster was down, is to play catch up and execute all the missed time slots until things are back on schedule.
You can override this behavior by setting ``catch_up`` to ``False``. This will make those schedules run only once when the cluster starts and normal scheduling resumes.
Defaults to ``True``.

.. _redis_configuration:

redis
~~~~~

Connection settings for Redis. Defaults::

    # redis defaults
    Q_CLUSTER = {
        'redis': {
            'host': 'localhost',
            'port': 6379,
            'db': 0,
            'password': <PASSWORD>,
            'socket_timeout': None,
            'charset': 'utf-8',
            'errors': 'strict',
            'unix_socket_path': None
        }
    }

For more information on these settings please refer to the `Redis-py <https://github.com/andymccurdy/redis-py>`__ documentation

.. _django_redis:

django_redis
~~~~~~~~~~~~

If you are already using `django-redis <https://github.com/niwinz/django-redis>`__ for your caching, you can take advantage of its excellent connection backend by supplying the name
of the cache connection you want to use::

    # example django-redis connection
    Q_CLUSTER = {
        'name': 'DJRedis',
        'workers': 4,
        'timeout': 90,
        'django_redis': 'default'
    }



.. tip::
    Django Q uses your ``SECRET_KEY`` to encrypt task packages and prevent task crossover. So make sure you have it set up in your Django settings.

.. _disque_configuration:

disque_nodes
~~~~~~~~~~~~
If you want to use Disque as your broker, set this to a list of available Disque nodes and each cluster will randomly try to connect to them::

    # example disque connection
    Q_CLUSTER = {
        'name': 'DisqueBroker',
        'workers': 4,
        'timeout': 60,
        'retry': 60,
        'disque_nodes': ['127.0.0.1:7711', '127.0.0.1:7712']
    }


Django Q is also compatible with the `Tynd <https://disque.tynd.co/>`__  addon on `Heroku <https://heroku.com>`__::

    # example Tynd connection
    import os

    Q_CLUSTER = {
        'name': 'TyndBroker',
        'workers': 8,
        'timeout': 30,
        'retry': 60,
        'disque_nodes': os.environ['TYND_DISQUE_NODES'].split(','),
        'disque_auth': os.environ['TYND_DISQUE_AUTH']
    }


disque_auth
~~~~~~~~~~~

Optional Disque password for servers that require authentication.

.. _ironmq_configuration:

iron_mq
~~~~~~~
Connection settings for IronMQ::<commit_msg>docs: minor changes<commit_after>Configuration
-------------
.. py:currentmodule:: django_q

Configuration is handled via the ``Q_CLUSTER`` dictionary in your :file:`settings.py`

.. code:: python

    # settings.py example
    Q_CLUSTER = {
        'name': 'myproject',
        'workers': 8,
        'recycle': 500,
        'timeout': 60,
        'compress': True,
        'save_limit': 250,
        'queue_limit': 500,
        'cpu_affinity': 1,
        'label': 'Django Q',
        'redis': {
            'host': '127.0.0.1',
            'port': 6379,
            'db': 0, }
    }

All configuration settings are optional:

name
~~~~

Used to differentiate between projects using the same Redis server. Defaults to ``'default'``.
This can be useful if you have several projects using the same Redis server.

.. note::
    Tasks are encrypted. When a worker encounters a task it can not decrypt, it will be discarded.

workers
~~~~~~~

The number of workers to use in the cluster. Defaults to CPU count of the current host, but can be set to a custom number.  [#f1]_

recycle
~~~~~~~

The number of tasks a worker will process before recycling . Useful to release memory resources on a regular basis. Defaults to ``500``.

.. _timeout:

timeout
~~~~~~~

The number of seconds a worker is allowed to spend on a task before it's terminated. Defaults to ``None``, meaning it will never time out.
Set this to something that makes sense for your project. Can be overridden for individual tasks.

.. _retry:

retry
~~~~~

The number of seconds a broker will wait for a cluster to finish a task, before it's presented again.
Only works with brokers that support delivery receipts. Defaults to 60 seconds.

compress
~~~~~~~~

Compresses task packages to Redis. Useful for large payloads, but can add overhead when used with many small packages.
Defaults to ``False``

.. _save_limit:

save_limit
~~~~~~~~~~

Limits the amount of successful tasks saved to Django.
 - Set to ``0`` for unlimited.
 - Set to ``-1`` for no success storage at all.
 - Defaults to ``250``
 - Failures are always saved.

.. _sync:

sync
~~~~

When set to ``True`` this configuration option forces all :func:`async` calls to be run with ``sync=True``.
Effectively making everything synchronous. Useful for testing. Defaults to ``False``.

.. _queue_limit:

queue_limit
~~~~~~~~~~~

This does not limit the amount of tasks that can be queued on the broker, but rather how many tasks are kept in memory by a single cluster.
Setting this to a reasonable number, can help balance the workload and the memory overhead of each individual cluster.
Defaults to ``workers**2``.

label
~~~~~

The label used for the Django Admin page. Defaults to ``'Django Q'``

.. _catch_up:

catch_up
~~~~~~~~
The default behavior for schedules that didn't run while a cluster was down, is to play catch up and execute all the missed time slots until things are back on schedule.
You can override this behavior by setting ``catch_up`` to ``False``. This will make those schedules run only once when the cluster starts and normal scheduling resumes.
Defaults to ``True``.

.. _redis_configuration:

redis
~~~~~

Connection settings for Redis. Defaults::

    # redis defaults
    Q_CLUSTER = {
        'redis': {
            'host': 'localhost',
            'port': 6379,
            'db': 0,
            'password': <PASSWORD>,
            'socket_timeout': None,
            'charset': 'utf-8',
            'errors': 'strict',
            'unix_socket_path': None
        }
    }

For more information on these settings please refer to the `Redis-py <https://github.com/andymccurdy/redis-py>`__ documentation

.. _django_redis:

django_redis
~~~~~~~~~~~~

If you are already using `django-redis <https://github.com/niwinz/django-redis>`__ for your caching, you can take advantage of its excellent connection backend by supplying the name
of the cache connection you want to use instead of a direct Redis connection::

    # example django-redis connection
    Q_CLUSTER = {
        'name': 'DJRedis',
        'workers': 4,
        'timeout': 90,
        'django_redis': 'default'
    }



.. tip::
    Django Q uses your ``SECRET_KEY`` to encrypt task packages and prevent task crossover. So make sure you have it set up in your Django settings.

.. _disque_configuration:

disque_nodes
~~~~~~~~~~~~
If you want to use Disque as your broker, set this to a list of available Disque nodes and each cluster will randomly try to connect to them::

    # example disque connection
    Q_CLUSTER = {
        'name': 'DisqueBroker',
        'workers': 4,
        'timeout': 60,
        'retry': 60,
        'disque_nodes': ['127.0.0.1:7711', '127.0.0.1:7712']
    }


Django Q is also compatible with the `Tynd Disque <https://disque.tynd.co/>`__  addon on `Heroku <https://heroku.com>`__::

    # example Tynd Disque connection
    import os

    Q_CLUSTER = {
        'name': 'TyndBroker',
        'workers': 8,
        'timeout': 30,
        'retry': 60,
        'bulk': 10,
        'disque_nodes': os.environ['TYND_DISQUE_NODES'].split(','),
        'disque_auth': os.environ['TYND_DISQUE_AUTH']
    }


disque_auth
~~~~~~~~~~~

Optional Disque password for servers that require authentication.

.. _ironmq_configuration:

iron_mq
~~~~~~~
Connection settings for IronMQ::
{'fraction_non_alphanumeric': 0.11574556830031282, 'fraction_numerical': 0.01440894871551806, 'mean_word_length': 3.0344168260038242, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 6, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4641485', 'n_tokens_mistral': 3208, 'n_tokens_neox': 3010, 'n_words': 1277}