<jupyter_start><jupyter_text>Getting started with Dask on Saturn Cloud Dask is a framework that easily lets you run Python in parallel across distributed machines. Below is a small example of using Dask on Saturn Cloud. The code creates a function that computes exponents and runs it across a list of inputs in parallel._For more details about the basics of Dask, read the [Parallelization in Python](https://saturncloud.io/docs/reference/dask_concepts/) article in the Saturn Cloud docs._ You can also look at the [Saturn Cloud Dask examples](https://saturncloud.io/docs/examples/dask/), and [the official Dask documentation](https://docs.dask.org/en/latest/).Before running this example, you need to [create a Dask cluster](https://saturncloud.io/docs/using-saturn-cloud/create-cluster/) associated with this resourceThis code chunk imports the Dask libraries and connects to the Saturn Cloud Dask cluster.<jupyter_code>import dask
from dask_saturn import SaturnCluster
from dask.distributed import Client

cluster = SaturnCluster()
client = Client(cluster)<jupyter_output><empty_output><jupyter_text>You can use the `@dask.delayed` decorator to change a regular Python function into a lazily-evaluated function. That means that a function call will return a future, instead of a value. The function won't immediately do the computation when it is run, instead only when that future object has its result requested.<jupyter_code>@dask.delayed
def lazy_exponent(args):
    x, y = args
    """Define a lazily evaluating function"""
    return x ** y<jupyter_output><empty_output><jupyter_text>The Dask distributed client comes with several methods for managing collections of such futures. The code below shows how to use a few of these.* `client.map(f, x)` - run function `f` once per item in a list-like object `x`. Returns a list of futures that can then be evaluated.* `client.gather(futures)` - Given a list of futures held by Dask workers, pulls them back to the client. Returns a list of delayed results (in this example).* `client.compute(delayed_results)` - Given a list of delayed results, brings the results on the client. Returns a list of futures (`sync=False`) or actual function results (`sync=True`)* `.result()` - Wait for a future to complete. Returns its actual value.All together, the code below will take the list of inputs, converts them to futures held by Dask workers for apply the exponental function, gathers them onto the client machine, and determines the results.<jupyter_code>inputs = [[1, 2], [3, 4], [5, 6], [9, 10], [11, 12]]

example_future = client.map(lazy_exponent, inputs)
futures_gathered = client.gather(example_future)
futures_computed = client.compute(futures_gathered, sync=False)

results = [x.result() for x in futures_computed]
results<jupyter_output><empty_output><jupyter_text>This was _somewhat_ of a toy example--you probably want to do more complex computations than exponents with Dask. However the core concept of making a function and then running it in a distributed fashion is at the core of what you can do with Dask on Saturn Cloud.When you're done, you can close the connection to the cluster:<jupyter_code>client.close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07538940809968847, 'fraction_numerical': 0.004049844236760125, 'mean_word_length': 5.296078431372549, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '406697', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 899, 'n_tokens_neox': 858, 'n_words': 410, 'path': 'examples/dask/start-with-dask.ipynb', 'repo_name': 'saturncloud/examples', 'size': '3210'}