<commit_before>:mod:`concurrent.futures` --- Launching parallel tasks
======================================================

.. module:: concurrent.futures
   :synopsis: Execute computations concurrently using threads or processes.

.. versionadded:: 3.2

**Source code:** :source:`Lib/concurrent/futures/thread.py`
and :source:`Lib/concurrent/futures/process.py`

--------------

The :mod:`concurrent.futures` module provides a high-level interface for
asynchronously executing callables.

The asynchronous execution can be performed with threads, using
:class:`ThreadPoolExecutor`, or separate processes, using
:class:`ProcessPoolExecutor`.  Both implement the same interface, which is
defined by the abstract :class:`Executor` class.


Executor Objects
----------------

.. class:: Executor

   An abstract class that provides methods to execute calls asynchronously.  It
   should not be used directly, but through its concrete subclasses.

    .. method:: submit(fn, *args, **kwargs)

       Schedules the callable, *fn*, to be executed as ``fn(*args **kwargs)``
       and returns a :class:`Future` object representing the execution of the
       callable. ::

          with ThreadPoolExecutor(max_workers=1) as executor:
              future = executor.submit(pow, 323, 1235)
              print(future.result())

    .. method:: map(func, *iterables, timeout=None)

       Equivalent to ``map(func, *iterables)`` except *func* is executed
       asynchronously and several calls to *func* may be made concurrently.  The
       returned iterator raises a :exc:`TimeoutError` if
       :meth:`~iterator.__next__` is called and the result isn't available
       after *timeout* seconds from the original call to :meth:`Executor.map`.
       *timeout* can be an int or a float.  If *timeout* is not specified or
       ``None``, there is no limit to the wait time.  If a call raises an
       exception, then that exception will be raised when its value is
       retrieved from the iterator.

    .. method:: shutdown(wait=True)

       Signal the executor that it should free any resources that it is using
       when the currently pending futures are done executing.  Calls to
       :meth:`Executor.submit` and :meth:`Executor.map` made after shutdown will
       raise :exc:`RuntimeError`.

       If *wait* is ``True`` then this method will not return until all the
       pending futures are done executing and the resources associated with the
       executor have been freed.  If *wait* is ``False`` then this method will
       return immediately and the resources associated with the executor will be
       freed when all pending futures are done executing.  Regardless of the
       value of *wait*, the entire Python program will not exit until all
       pending futures are done executing.

       You can avoid having to call this method explicitly if you use the
       :keyword:`with` statement, which will shutdown the :class:`Executor`
       (waiting as if :meth:`Executor.shutdown` were called with *wait* set to
       ``True``)::

          import shutil
          with ThreadPoolExecutor(max_workers=4) as e:
              e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
              e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
              e.submit(shutil.copy, 'src3.txt', 'dest3.txt')
              e.submit(shutil.copy, 'src3.txt', 'dest4.txt')


ThreadPoolExecutor
------------------

:class:`ThreadPoolExecutor` is a :class:`Executor` subclass that uses a pool of
threads to execute calls asynchronously.

Deadlocks can occur when the callable associated with a :class:`Future` waits on
the results of another :class:`Future`.  For example::

   import time
   def wait_on_b():
       time.sleep(5)
       print(b.result()) # b will never complete because it is waiting on a.
       return 5

   def wait_on_a():
       time.sleep(5)
       print(a.result()) # a will never complete because it is waiting on b.
       return 6


   executor = ThreadPoolExecutor(max_workers=2)
   a = executor.submit(wait_on_b)
   b = executor.submit(wait_on_a)

And::

   def wait_on_future():
       f = executor.submit(pow, 5, 2)
       # This will never complete because there is only one worker thread and
       # it is executing this function.
       print(f.result())

   executor = ThreadPoolExecutor(max_workers=1)
   executor.submit(wait_on_future)


.. class:: ThreadPoolExecutor(max_workers)

   An :class:`Executor` subclass that uses a pool of at most *max_workers*
   threads to execute calls asynchronously.


.. _threadpoolexecutor-example:

ThreadPoolExecutor Example
~~~~~~~~~~~~~~~~~~~~~~~~~~
::

   import concurrent.futures
   import urllib.request

   URLS = ['http://www.foxnews.com/',
           'http://www.cnn.com/',
           'http://europe.wsj.com/',
           'http://www.bbc.co.uk/',
           'http://some-made-up-domain.com/']

   # Retrieve a single page and report the url and contents
   def load_url(url, timeout):
       conn = urllib.request.urlopen(url, timeout=timeout)
       return conn.readall()

   # We can use a with statement to ensure threads are cleaned up promptly
   with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
       # Start the load operations and mark each future with its URL
       future_to_url = {executor.submit(load_url, url, 60):url for url in URLS}
       for future in concurrent.futures.as_completed(future_to_url):
           url = future_to_url[url]
           try:
               data = future.result()
           except Exception as exc:
               print('%r generated an exception: %s' % (url, exc))
           else:
               print('%r page is %d bytes' % (url, len(data)))

<commit_msg>Fix typo in example (pointed out by a couple of people, most recently <NAME>)<commit_after>:mod:`concurrent.futures` --- Launching parallel tasks
======================================================

.. module:: concurrent.futures
   :synopsis: Execute computations concurrently using threads or processes.

.. versionadded:: 3.2

**Source code:** :source:`Lib/concurrent/futures/thread.py`
and :source:`Lib/concurrent/futures/process.py`

--------------

The :mod:`concurrent.futures` module provides a high-level interface for
asynchronously executing callables.

The asynchronous execution can be performed with threads, using
:class:`ThreadPoolExecutor`, or separate processes, using
:class:`ProcessPoolExecutor`.  Both implement the same interface, which is
defined by the abstract :class:`Executor` class.


Executor Objects
----------------

.. class:: Executor

   An abstract class that provides methods to execute calls asynchronously.  It
   should not be used directly, but through its concrete subclasses.

    .. method:: submit(fn, *args, **kwargs)

       Schedules the callable, *fn*, to be executed as ``fn(*args **kwargs)``
       and returns a :class:`Future` object representing the execution of the
       callable. ::

          with ThreadPoolExecutor(max_workers=1) as executor:
              future = executor.submit(pow, 323, 1235)
              print(future.result())

    .. method:: map(func, *iterables, timeout=None)

       Equivalent to ``map(func, *iterables)`` except *func* is executed
       asynchronously and several calls to *func* may be made concurrently.  The
       returned iterator raises a :exc:`TimeoutError` if
       :meth:`~iterator.__next__` is called and the result isn't available
       after *timeout* seconds from the original call to :meth:`Executor.map`.
       *timeout* can be an int or a float.  If *timeout* is not specified or
       ``None``, there is no limit to the wait time.  If a call raises an
       exception, then that exception will be raised when its value is
       retrieved from the iterator.

    .. method:: shutdown(wait=True)

       Signal the executor that it should free any resources that it is using
       when the currently pending futures are done executing.  Calls to
       :meth:`Executor.submit` and :meth:`Executor.map` made after shutdown will
       raise :exc:`RuntimeError`.

       If *wait* is ``True`` then this method will not return until all the
       pending futures are done executing and the resources associated with the
       executor have been freed.  If *wait* is ``False`` then this method will
       return immediately and the resources associated with the executor will be
       freed when all pending futures are done executing.  Regardless of the
       value of *wait*, the entire Python program will not exit until all
       pending futures are done executing.

       You can avoid having to call this method explicitly if you use the
       :keyword:`with` statement, which will shutdown the :class:`Executor`
       (waiting as if :meth:`Executor.shutdown` were called with *wait* set to
       ``True``)::

          import shutil
          with ThreadPoolExecutor(max_workers=4) as e:
              e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
              e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
              e.submit(shutil.copy, 'src3.txt', 'dest3.txt')
              e.submit(shutil.copy, 'src3.txt', 'dest4.txt')


ThreadPoolExecutor
------------------

:class:`ThreadPoolExecutor` is a :class:`Executor` subclass that uses a pool of
threads to execute calls asynchronously.

Deadlocks can occur when the callable associated with a :class:`Future` waits on
the results of another :class:`Future`.  For example::

   import time
   def wait_on_b():
       time.sleep(5)
       print(b.result()) # b will never complete because it is waiting on a.
       return 5

   def wait_on_a():
       time.sleep(5)
       print(a.result()) # a will never complete because it is waiting on b.
       return 6


   executor = ThreadPoolExecutor(max_workers=2)
   a = executor.submit(wait_on_b)
   b = executor.submit(wait_on_a)

And::

   def wait_on_future():
       f = executor.submit(pow, 5, 2)
       # This will never complete because there is only one worker thread and
       # it is executing this function.
       print(f.result())

   executor = ThreadPoolExecutor(max_workers=1)
   executor.submit(wait_on_future)


.. class:: ThreadPoolExecutor(max_workers)

   An :class:`Executor` subclass that uses a pool of at most *max_workers*
   threads to execute calls asynchronously.


.. _threadpoolexecutor-example:

ThreadPoolExecutor Example
~~~~~~~~~~~~~~~~~~~~~~~~~~
::

   import concurrent.futures
   import urllib.request

   URLS = ['http://www.foxnews.com/',
           'http://www.cnn.com/',
           'http://europe.wsj.com/',
           'http://www.bbc.co.uk/',
           'http://some-made-up-domain.com/']

   # Retrieve a single page and report the url and contents
   def load_url(url, timeout):
       conn = urllib.request.urlopen(url, timeout=timeout)
       return conn.readall()

   # We can use a with statement to ensure threads are cleaned up promptly
   with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
       # Start the load operations and mark each future with its URL
       future_to_url = {executor.submit(load_url, url, 60):url for url in URLS}
       for future in concurrent.futures.as_completed(future_to_url):
           url = future_to_url[future]
           try:
               data = future.result()
           except Exception as exc:
               print('%r generated an exception: %s' % (url, exc))
           else:
               print('%r page is %d bytes' % (url, len(data)))


{'fraction_non_alphanumeric': 0.11298306556665219, 'fraction_numerical': 0.005210594876248371, 'mean_word_length': 2.9250170415814587, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3498359', 'n_tokens_mistral': 3258, 'n_tokens_neox': 3086, 'n_words': 1177}