<jupyter_start><jupyter_text>A Guided Tour of Ray Core: Multiprocessing Pool [*Distributed multiprocessing.Pool*](https://docs.ray.io/en/latest/multiprocessing.html) make it easy to scale existing Python applications that use [`multiprocessing.Pool`](https://docs.python.org/3/library/multiprocessing.html) by leveraging *actors*.--- First, let's start Ray…<jupyter_code>import logging
import ray

ray.init(
    ignore_reinit_error=True,
    logging_level=logging.ERROR,
)<jupyter_output><empty_output><jupyter_text>Multiprocessing Pool example The following is a simple Python function with a slight delay added (to make it behave like a more complex calculation)...<jupyter_code>import time

def my_function (x):
    time.sleep(1)
    return x ** 2
%%time

my_function(11)<jupyter_output><empty_output><jupyter_text>First, let's try using this with Python's built-in [multiprocessing.Pool](https://docs.python.org/3/library/multiprocessing.html):<jupyter_code>%%time

from multiprocessing import Pool

pool = Pool()

for result in pool.map(my_function, range(50)):
    print(result)<jupyter_output><empty_output><jupyter_text>Now we'll create a *Pool* using and distribute its tasks across a cluster (or across the available cores on a laptop):<jupyter_code>%%time

from ray.util.multiprocessing import Pool

pool = Pool()

for result in pool.map(my_function, range(50)):
    print(result)<jupyter_output><empty_output><jupyter_text>The distributed version has the trade-off of increased overhead, although now it can scale-out horizontally across a cluster. The benefits would be more pronounced with a more computationally expensive calculation. Finally, shutdown Ray<jupyter_code>ray.shutdown()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10005783689994216, 'fraction_numerical': 0.00578368999421631, 'mean_word_length': 6.361702127659575, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '510816', 'license': 'MIT', 'n_tokens_mistral': 552, 'n_tokens_neox': 529, 'n_words': 152, 'path': 'ex_04_mult_pool.ipynb', 'repo_name': 'ArasAzimi/ray_tutorial', 'size': '1729'}