<commit_before>Basic Model and Quickstart
==========================

Install
-------

::

    $ pip install distributed

Setup Cluster
-------------

Set up center and worker nodes on your local computer with the ``dcluster``
command::

   $ dcenter 127.0.0.1 127.0.0.1 127.0.0.1 127.0.0.1

See :doc:`setup <setup>` for more advanced use.

Launch Executor
---------------

Launch an Executor to interact with the network.  Provide the first address
listed in the ``dcenter`` call with the port ``8787``.

.. code-block:: python

   >>> from distributed import Executor
   >>> executor = Executor('127.0.0.1:8787')

Map and Submit
~~~~~~~~~~~~~~

The executor provides ``map`` and ``submit`` functions like
``concurrent.futures.Executor``.  Results of these functions are ``Future``
objects, proxies for data that lives on the cluster.

.. code-block:: python

   >>> def square(x):
           return x ** 2

   >>> def neg(x):
           return -x

   >>> A = executor.map(square, range(10))
   >>> B = executor.map(neg, A)
   >>> total = executor.submit(sum, B)
   >>> total.result()
   -285

Gather
~~~~~~

Gather results to your local machine either with the ``Future.result`` method
as shown below for a single future, or with the ``Executor.gather`` method for
many futures at once.

.. code-block:: python

   >>> executor.gather(A)
   [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

get
~~~

The ``Executor.get`` method operates like a typical dask scheduler.

Get works with raw dask graphs:

.. code-block:: python

   >>> dsk = {'a': 1, 'b': (inc, 'a')}
   >>> executor.get(dsk, 'b')
   2

Get works with dask collections (like dask.array or dask.dataframe):

.. code-block:: python

   >>> import dask.array as da
   >>> x = da.arange(10, chunks=(5,))
   >>> x.sum().compute(get=executor.get)
   45

See :doc:`executor <executor>` for more advanced use.<commit_msg>minor fixes to quickstart<commit_after>Basic Model and Quickstart
==========================

Install
-------

::

    $ pip install distributed

Setup Cluster
-------------

Set up center and worker nodes on your local computer with the ``dcluster``
command::

   $ dcluster 127.0.0.1 127.0.0.1 127.0.0.1 127.0.0.1

See :doc:`setup <setup>` for more advanced use.

Launch Executor
---------------

Launch an Executor to interact with the network.  Provide the first address
listed in the ``dcenter`` call with the port ``8787``.

.. code-block:: python

   >>> from distributed import Executor
   >>> executor = Executor('127.0.0.1:8787')

Map and Submit
~~~~~~~~~~~~~~

The executor provides ``map`` and ``submit`` functions like
``concurrent.futures.Executor``.  Results of these functions are ``Future``
objects, proxies for data that lives on the cluster.

.. code-block:: python

   >>> def square(x):
           return x ** 2

   >>> def neg(x):
           return -x

   >>> A = executor.map(square, range(10))
   >>> B = executor.map(neg, A)
   >>> total = executor.submit(sum, B)
   >>> total.result()
   -285

Gather
~~~~~~

Gather results to your local machine either with the ``Future.result`` method
as shown below for a single future, or with the ``Executor.gather`` method for
many futures at once.

.. code-block:: python

   >>> executor.gather(A)
   [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

get
~~~

The ``Executor.get`` method operates like a typical dask scheduler.

Get works with raw dask graphs:

.. code-block:: python

   >>> dsk = {'a': 12, 'b': (square, 'a')}
   >>> executor.get(dsk, 'b')
   144

Get works with dask collections (like dask.array or dask.dataframe):

.. code-block:: python

   >>> import dask.array as da
   >>> x = da.arange(10, chunks=(5,))
   >>> x.sum().compute(get=executor.get)
   45

See :doc:`executor <executor>` for more advanced use.
{'fraction_non_alphanumeric': 0.1795896616040501, 'fraction_numerical': 0.036504130029309884, 'mean_word_length': 3.1072210065645516, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 91, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 10, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6581758', 'n_tokens_mistral': 1307, 'n_tokens_neox': 1191, 'n_words': 426}