<commit_before>Furious - An Asynchronous Workflow Library.
===========================================

Introduction
------------

Furious is a lightweight library that wraps Google App Engine taskqueues to
make building dynamic workflows easy.


Installation
------------

Include `furious` within your App Engine project's libary directory, often
`/lib`.  If that does not make sense to you, simply include the `furious`
subdirectory within your application's root directory (where your `app.yaml`
is located).

Usage
-----

In the simplest form, usage looks like:

    from furious import Async

    # Create an Async object.
    async = Async(
        target="your.module.func",
        args=("positstional", "args"),
        kwargs={"kwargs": "too"})

    # Tell the async to insert itself to be run.
    async.start()

This inserts a task that will make the following call:

    your.module.func("pos", "args", kwarg="too")


### Grouping async jobs

You can group jobs together,

    from furious import context

    # Instantiate a new Context.
    with context.new() as batch:

        for number in xrange(10):

            # Create a new Async object that will be added to the Context.
            batch.add(target="square_a_number", args=(number,))


### Setting defaults.

It is possible to set options, like the target queue,

    from furious import context, defaults

    @defaults(queue='square')
    def square_a_number(number):
        return number * number

    @defaults(queue='delayed_square', task_args={'countdown': 20})
    def delayed_square(number):
        return number * number

    # Instantiate a new Context.
    with context.new() as batch:

        for number in xrange(10):

            if number % 2:
                # At insert time, the task is added to the 'square' queue.
                batch.add(target="square_a_number", args=(number,))
            else:
                # At insert time, the task is added to the 'delayed_square'
                # queue with a 20 second countdown.
                batch.add(target="delayed_square", args=(number,))


Tasks targeted at the same queue will be batch inserted.


### Workflows via Contexts

*NOTE*: The Context on__complete method is not yet fully implemented.

Contexts allow you to build workflows easily,

    from furious import Async
    from furious.context import get_current_context, new

    def square_a_number(number):
        return number ** number

    def sum_results():
        # Get the current context.
        context = get_current_context()

        # Get an iterator that will iterate over the results.
        results = context.get_results()

        # Do something with the results.
        return sum(result.payload for result in results)

    # Instantiate a new Context.
    with context.new() as batch:

        # Set a function to be called when all async jobs are complete.
        batch.on_complete(sum_results)

        for number in xrange(10):
<commit_msg>Removed extra _ in on_complete note.<commit_after>Furious - An Asynchronous Workflow Library.
===========================================

Introduction
------------

Furious is a lightweight library that wraps Google App Engine taskqueues to
make building dynamic workflows easy.


Installation
------------

Include `furious` within your App Engine project's libary directory, often
`/lib`.  If that does not make sense to you, simply include the `furious`
subdirectory within your application's root directory (where your `app.yaml`
is located).

Usage
-----

In the simplest form, usage looks like:

    from furious import Async

    # Create an Async object.
    async = Async(
        target="your.module.func",
        args=("positstional", "args"),
        kwargs={"kwargs": "too"})

    # Tell the async to insert itself to be run.
    async.start()

This inserts a task that will make the following call:

    your.module.func("pos", "args", kwarg="too")


### Grouping async jobs

You can group jobs together,

    from furious import context

    # Instantiate a new Context.
    with context.new() as batch:

        for number in xrange(10):

            # Create a new Async object that will be added to the Context.
            batch.add(target="square_a_number", args=(number,))


### Setting defaults.

It is possible to set options, like the target queue,

    from furious import context, defaults

    @defaults(queue='square')
    def square_a_number(number):
        return number * number

    @defaults(queue='delayed_square', task_args={'countdown': 20})
    def delayed_square(number):
        return number * number

    # Instantiate a new Context.
    with context.new() as batch:

        for number in xrange(10):

            if number % 2:
                # At insert time, the task is added to the 'square' queue.
                batch.add(target="square_a_number", args=(number,))
            else:
                # At insert time, the task is added to the 'delayed_square'
                # queue with a 20 second countdown.
                batch.add(target="delayed_square", args=(number,))


Tasks targeted at the same queue will be batch inserted.


### Workflows via Contexts

*NOTE*: The Context on_complete method is not yet fully implemented.

Contexts allow you to build workflows easily,

    from furious import Async
    from furious.context import get_current_context, new

    def square_a_number(number):
        return number ** number

    def sum_results():
        # Get the current context.
        context = get_current_context()

        # Get an iterator that will iterate over the results.
        results = context.get_results()

        # Do something with the results.
        return sum(result.payload for result in results)

    # Instantiate a new Context.
    with context.new() as batch:

        # Set a function to be called when all async jobs are complete.
        batch.on_complete(sum_results)

        for number in xrange(10):

{'fraction_non_alphanumeric': 0.10006682258603408, 'fraction_numerical': 0.0036752422318743734, 'mean_word_length': 2.688847812692545, 'pattern_counts': {'":': 2, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3032403', 'n_tokens_mistral': 1639, 'n_tokens_neox': 1518, 'n_words': 642}