<commit_before>
:mod:`futures` --- Asynchronous computation
===========================================

.. module:: futures
   :synopsis: Execute computations asynchronously using threads or
   (experimentally) processes. 

XXX Overview. XXX Make this documentation suck less.

Executor Objects
----------------

:class:`Executor` is an abstract class that provides methods to execute calls
asynchronously. It should not be used directly, but through its two
subclasses: :class:`ThreadPoolExecutor` and (experimental)
:class:`ProcessPoolExecutor`.

.. method:: Executor.run_to_futures(calls, timeout=None, return_when=ALL_COMPLETED)

   Schedule the given calls for execution and return a :class:`FutureList`
   containing a :class:`Future` for each call. This method should always be
   called using keyword arguments, which are:

   *calls* must be a sequence of callables that take no arguments.

   *timeout* can be used to control the maximum number of seconds to wait before
   returning. If *timeout* is not specified or ``None`` then there is no limit
   to the wait time.

   *return_when* indicates when the method should return. It must be one of the
   following constants:

      +-----------------------------+----------------------------------------+
      | Constant                    | Description                            |
      +=============================+========================================+
      | :const:`FIRST_COMPLETED`    | The method will return when any call   |
      |                             | finishes.                              |
      +-----------------------------+----------------------------------------+
      | :const:`FIRST_EXCEPTION`    | The method will return when any call   |
      |                             | raises an exception or when all calls  |
      |                             | finish.                                |
      +-----------------------------+----------------------------------------+
      | :const:`ALL_COMPLETED`      | The method will return when all calls  |
      |                             | finish.                                |
      +-----------------------------+----------------------------------------+
      | :const:`RETURN_IMMEDIATELY` | The method will return immediately.    |
      +-----------------------------+----------------------------------------+

.. method:: Executor.run_to_results(calls, timeout=None)

   Schedule the given calls for execution and return an iterator over their
   results. Raises a :exc:`TimeoutError` if the calls do not complete before 
   *timeout* seconds. If *timeout* is not specified or ``None`` then there is no
   limit to the wait time.

.. method:: Executor.map(func, *iterables, timeout=None)

   Equivalent to map(*func*, *\*iterables*) but executed asynchronously. Raises
   a :exc:`TimeoutError` if the map cannot be generated before *timeout*
   seconds. If *timeout* is not specified or ``None`` then there is no limit to
   the wait time.

.. method:: Executor.shutdown()

   Signal the executor that it should free any resources that it is using when
   the currently pending futures are done executing. Calls to
   :meth:`Executor.run_to_futures`, :meth:`Executor.run_to_results` and
   :meth:`Executor.map` made after shutdown will raise :exc:`RuntimeError`.

ThreadPoolExecutor Objects
--------------------------

The :class:`ThreadPoolExecutor` class is an :class:`Executor` subclass that uses
a pool of threads to execute calls asynchronously.

.. class:: ThreadPoolExecutor(max_threads)

   Executes calls asynchronously using at pool of at most *max_threads* threads.

.. _threadpoolexecutor-example:

ThreadPoolExecutor Example
^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   import functools
   import urllib.request
   import futures
   
   URLS = ['http://www.foxnews.com/',
           'http://www.cnn.com/',
           'http://europe.wsj.com/',
           'http://www.bbc.co.uk/',
           'http://some-made-up-domain.com/']
   
   def load_url(url, timeout):
       return urllib.request.urlopen(url, timeout=timeout).read()
   
   with futures.ThreadPoolExecutor(50) as executor:
      future_list = executor.run_to_futures(
              [functools.partial(load_url, url, 30) for url in URLS])
   
   for url, future in zip(URLS, future_list):
       if future.exception() is not None:
           print('%r generated an exception: %s' % (url, future.exception()))
       else:
           print('%r page is %d bytes' % (url, len(future.result())))

ProcessPoolExecutor Objects
---------------------------

The :class:`ProcessPoolExecutor` class is an **experimental** :class:`Executor`
subclass that uses a pool of processes to execute calls asynchronously. There
are situations where it can deadlock.

.. class:: ProcessPoolExecutor(max_processes=None)

   Executes calls asynchronously using a pool of at most *max_processes*
   processes. If *max_processes* is ``None`` or not given then as many worker
   processes will be created as the machine has processors.

XXX Same as ThreadPoolExecutor except:
   * serialization limits
   * no GIL issues

ProcessPoolExecutor Example
^^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   PRIMES = [
       112272535095293,
       112582705942171,
       112272535095293,
       115280095190773,
       115797848077099,
       1099726899285419]

   def is_prime(n):
       if n % 2 == 0:
           return False

       sqrt_n = int(math.floor(math.sqrt(n)))
       for i in range(3, sqrt_n + 1, 2):
           if n % i == 0:
               return False
       return True

   with futures.ProcessPoolExecutor() as executor:
       for number, is_prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
           print('%d is prime: %s' % (number, is_prime))

FutureList Objects
------------------

The :class:`FutureList` class is an immutable container for :class:`Future`
instances and should only be instantiated by :meth:`Executor.run_to_futures`.

.. method:: FutureList.wait(timeout=None, return_when=ALL_COMPLETED)

   XXX Copy this from ThreadPoolExecutor but with a note about RETURN_IMMEDIATELY
   being useless

.. method:: FutureList.cancel(timeout=None)<commit_msg>Fixed all XXX comments and added a module overview.<commit_after>:mod:`futures` --- Asynchronous computation
===========================================

.. module:: futures
   :synopsis: Execute computations asynchronously using threads or
   (experimentally) processes. 

The :mod:`futures` module provides a high-level interface for asynchronously
executing functions and methods.

The asynchronous execution can be be performed by threads, using
:class:`ThreadPoolExecutor`, or seperate processes, using
:class:`ProcessPoolExecutor`. Both implement the same interface, which is
defined by the abstract :class:`Executor` class.

Executor Objects
----------------

:class:`Executor` is an abstract class that provides methods to execute calls
asynchronously. It should not be used directly, but through its two
subclasses: :class:`ThreadPoolExecutor` and (experimental)
:class:`ProcessPoolExecutor`.

.. method:: Executor.run_to_futures(calls, timeout=None, return_when=ALL_COMPLETED)

   Schedule the given calls for execution and return a :class:`FutureList`
   containing a :class:`Future` for each call. This method should always be
   called using keyword arguments, which are:

   *calls* must be a sequence of callables that take no arguments.

   *timeout* can be used to control the maximum number of seconds to wait before
   returning. If *timeout* is not specified or ``None`` then there is no limit
   to the wait time.

   *return_when* indicates when the method should return. It must be one of the
   following constants:

      +-----------------------------+----------------------------------------+
      | Constant                    | Description                            |
      +=============================+========================================+
      | :const:`FIRST_COMPLETED`    | The method will return when any call   |
      |                             | finishes.                              |
      +-----------------------------+----------------------------------------+
      | :const:`FIRST_EXCEPTION`    | The method will return when any call   |
      |                             | raises an exception or when all calls  |
      |                             | finish.                                |
      +-----------------------------+----------------------------------------+
      | :const:`ALL_COMPLETED`      | The method will return when all calls  |
      |                             | finish.                                |
      +-----------------------------+----------------------------------------+
      | :const:`RETURN_IMMEDIATELY` | The method will return immediately.    |
      +-----------------------------+----------------------------------------+

.. method:: Executor.run_to_results(calls, timeout=None)

   Schedule the given calls for execution and return an iterator over their
   results. Raises a :exc:`TimeoutError` if the calls do not complete before 
   *timeout* seconds. If *timeout* is not specified or ``None`` then there is no
   limit to the wait time.

.. method:: Executor.map(func, *iterables, timeout=None)

   Equivalent to map(*func*, *\*iterables*) but executed asynchronously. Raises
   a :exc:`TimeoutError` if the map cannot be generated before *timeout*
   seconds. If *timeout* is not specified or ``None`` then there is no limit to
   the wait time.

.. method:: Executor.shutdown()

   Signal the executor that it should free any resources that it is using when
   the currently pending futures are done executing. Calls to
   :meth:`Executor.run_to_futures`, :meth:`Executor.run_to_results` and
   :meth:`Executor.map` made after shutdown will raise :exc:`RuntimeError`.

ThreadPoolExecutor Objects
--------------------------

The :class:`ThreadPoolExecutor` class is an :class:`Executor` subclass that uses
a pool of threads to execute calls asynchronously.

.. class:: ThreadPoolExecutor(max_threads)

   Executes calls asynchronously using at pool of at most *max_threads* threads.

.. _threadpoolexecutor-example:

ThreadPoolExecutor Example
^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   import functools
   import urllib.request
   import futures
   
   URLS = ['http://www.foxnews.com/',
           'http://www.cnn.com/',
           'http://europe.wsj.com/',
           'http://www.bbc.co.uk/',
           'http://some-made-up-domain.com/']
   
   def load_url(url, timeout):
       return urllib.request.urlopen(url, timeout=timeout).read()
   
   with futures.ThreadPoolExecutor(50) as executor:
      future_list = executor.run_to_futures(
              [functools.partial(load_url, url, 30) for url in URLS])
   
   for url, future in zip(URLS, future_list):
       if future.exception() is not None:
           print('%r generated an exception: %s' % (url, future.exception()))
       else:
           print('%r page is %d bytes' % (url, len(future.result())))

ProcessPoolExecutor Objects
---------------------------

The :class:`ProcessPoolExecutor` class is an **experimental** :class:`Executor`
subclass that uses a pool of processes to execute calls asynchronously. There
are situations where it can deadlock. :class:`ProcessPoolExecutor` uses the
:mod:`multiprocessing` module, which allows it to side-step the
:term:`Global Interpreter Lock` but also means that only picklable objects can
be executed and returned.

.. class:: ProcessPoolExecutor(max_processes=None)

   Executes calls asynchronously using a pool of at most *max_processes*
   processes. If *max_processes* is ``None`` or not given then as many worker
   processes will be created as the machine has processors.

ProcessPoolExecutor Example
^^^^^^^^^^^^^^^^^^^^^^^^^^^
::

   PRIMES = [
       112272535095293,
       112582705942171,
       112272535095293,
       115280095190773,
       115797848077099,
       1099726899285419]

   def is_prime(n):
       if n % 2 == 0:
           return False

       sqrt_n = int(math.floor(math.sqrt(n)))
       for i in range(3, sqrt_n + 1, 2):
           if n % i == 0:
               return False
       return True

   with futures.ProcessPoolExecutor() as executor:
       for number, is_prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
           print('%d is prime: %s' % (number, is_prime))

FutureList Objects
------------------

The :class:`FutureList` class is an immutable container for :class:`Future`
instances and should only be instantiated by :meth:`Executor.run_to_futures`.

.. method:: FutureList.wait(timeout=None, return_when=ALL_COMPLETED)

   Wait until the given conditions are met. This method should always be
   called using keyword arguments, which are:

   *timeout* can be used to control the maximum number of seconds to wait before
   returning. If *timeout* is not specified or ``None`` then there is no limit
   to the wait time.

   *return_when* indicates when the method should return. It must be one of the
   following constants:

      +-----------------------------+----------------------------------------+
      | Constant                    | Description                            |
      +=============================+========================================+
      | :const:`FIRST_COMPLETED`    | The method will return when any call   |
      |                             | finishes.                              |
      +-----------------------------+----------------------------------------+
      | :const:`FIRST_EXCEPTION`    | The method will return when any call   |
      |                             | raises an exception or when all calls  |
      |                             | finish.                                |
      +-----------------------------+----------------------------------------+
      | :const:`ALL_COMPLETED`      | The method will return when all calls  |
      |                             | finish.                                |
      +-----------------------------+----------------------------------------+
      | :const:`RETURN_IMMEDIATELY` | The method will return immediately.    |
      |                             | This option is only available for      |
      |                             | consistency with                       |
      |                             | :meth:`Executor.run_to_results` and is |
      |                             | not likely to be useful.               |
      +-----------------------------+----------------------------------------+

.. method:: FutureList.cancel(timeout=None)
{'fraction_non_alphanumeric': 0.1875342090859332, 'fraction_numerical': 0.013820470717022441, 'mean_word_length': 1.942230273752013, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 17, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1918214', 'n_tokens_mistral': 3843, 'n_tokens_neox': 3564, 'n_words': 1267}