<commit_before>Basic Model and Quickstart
==========================

Install
-------

::

    $ pip install distributed

Setup Network
-------------

Given a network of computers ``192.168.0.{100,101,102}`` we run the ``dcenter``
command on one of them::

   $ ssh 192.168.1.100
   $ dcenter
   Start center at 192.168.1.100:8787

We run ``dworker`` on the others, passing the address of the first as a command
line argument::

   $ ssh 192.168.1.101
   $ dworker 192.168.1.100:8787
   Start worker at:            192.168.1.101:8788
   Registered with center at:  192.168.1.100:8787

   $ ssh 192.168.1.102
   $ dworker 192.168.1.100:8787
   Start worker at:            192.168.1.102:8788
   Registered with center at:  192.168.1.100:8787

These processes talk to each other over the network.  Each worker serves
data and runs arbitrary functions on that data.  The workers share data with
each other.  The center knows all of the workers and what data they have.

If you want to do this on your local computer then omit the ``ssh`` commands
and replace the IP addresses above with ``127.0.0.1``.


User Clients
------------

We use our network with two client modules:

*  ``Pool`` - mimics ``multiprocessing.Pool``
*  ``get`` - mimics ``dask.get``.

Pool
````

The pool provides ``map``, ``apply``, and ``apply_async`` functions.  Results
of these functions are ``RemoteData`` objects, proxies that live on the<commit_msg>update doc<commit_after>Basic Model and Quickstart
==========================

Install
-------

::

    $ pip install distributed

Setup Network
-------------

Given a network of computers ``192.168.0.{100,101,102}`` we run the ``dcenter``
command on one of them::

   $ ssh 192.168.1.100
   $ dcenter
   Start center at 192.168.1.100:8787

We run ``dworker`` on the others, passing the address of the first as a command
line argument::

   $ ssh 192.168.1.101
   $ dworker 192.168.1.100:8787
   Start worker at:            192.168.1.101:8788
   Registered with center at:  192.168.1.100:8787

   $ ssh 192.168.1.102
   $ dworker 192.168.1.100:8787
   Start worker at:            192.168.1.102:8788
   Registered with center at:  192.168.1.100:8787

These processes talk to each other over the network.  Each worker serves
data and runs arbitrary functions on that data.  The workers share data with
each other.  The center knows all of the workers and what data they have.

If you want to do this on your local computer then omit the ``ssh`` commands
and replace the IP addresses above with ``127.0.0.1``::

   $ dcenter
   $ dworker 127.0.0.1:8787
   $ dworker 127.0.0.1:8787


User Clients
------------

We use our network with two client modules:

*  ``Pool`` - mimics ``multiprocessing.Pool``
*  ``get`` - mimics ``dask.get``.

Pool
````

The pool provides ``map``, ``apply``, and ``apply_async`` functions.  Results
of these functions are ``RemoteData`` objects, proxies that live on the
{'fraction_non_alphanumeric': 0.1405071967100754, 'fraction_numerical': 0.10966415352981494, 'mean_word_length': 2.9931600547195623, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 24, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6058232', 'n_tokens_mistral': 1132, 'n_tokens_neox': 902, 'n_words': 353}