<jupyter_start><jupyter_text>OPTaaS Batching Note: To run this notebook, you need an API Key. You can get one here.OPTaaS can facilitate parallel computation, where you generate a batch of configurations, pass them to a number of workers to calculate the results, and then store the results to get the next batch of configurations.  Connect to OPTaaS using your API Key<jupyter_code>from mindfoundry.optaas.client.client import OPTaaSClient

client = OPTaaSClient('https://optaas.mindfoundry.ai', '<Your OPTaaS API key>')<jupyter_output><empty_output><jupyter_text>Create your Task<jupyter_code>from mindfoundry.optaas.client.parameter import FloatParameter

task = client.create_task(
    title='Batching Example',
    parameters=[
        FloatParameter('x', minimum=0, maximum=1),
        FloatParameter('y', minimum=0.1, maximum=2)
    ]
)<jupyter_output><empty_output><jupyter_text>Set up your workersThis is just a simple example of how you would pass a Configuration to a worker and get a Result back. Your process will of course likely be more complex!The number of workers will depend on how much processing power you have available. In order to get the best quality configurations from OPTaaS, we recommend using no more than 10.<jupyter_code>from multiprocessing import Pool
from time import sleep
from mindfoundry.optaas.client.result import Result

number_of_workers = 4

def spin_off_workers_and_get_results(configurations):
    with Pool(number_of_workers) as pool:
        return pool.map(get_result, configurations)

def get_result(configuration):
    x = configuration.values['x']
    y = configuration.values['y']
    sleep(1)  # Calculating score...
    score = (x * y) - (x / y)
    return Result(configuration=configuration, score=score)<jupyter_output><empty_output><jupyter_text>Generate the first batch of configurations<jupyter_code>configurations = task.generate_configurations(number_of_workers)
display(configurations)<jupyter_output><empty_output><jupyter_text>Record results to get the next batch of configurationsThe next batch will be the same size as the number of results you record.<jupyter_code>number_of_batches = 3

for _ in range(number_of_batches):
    results = spin_off_workers_and_get_results(configurations)
    print(f"Scores: {[result.score for result in results]}\n")

    configurations = task.record_results(results)
    print(f"Next configurations: {[c.values for c in configurations]}\n")<jupyter_output>Scores: [0.04880952380952386, 0.32982698432350455, -3.5595532282188787, 0.19394652746118213]

Next configurations: [{'x': 0.6177680922768689, 'y': 0.450418600138205}, {'x': 0.08624546027300994, 'y': 1.810425236962426}, {'x': 0.2448349778331208, 'y': 0.3929881953445803}, {'x': 0.11298400991785473, 'y': 0.44078687252193205}]

Scores: [-1.0932879041023202, 0.10850272429629561, -0.5267912228698785, -0.20652157717292371]

Next configurations: [{'x': 0.12110706920922909, 'y': 0.9073183154180738}, {'x': 0.5814744623833986, 'y': 1.9081733977000617}, {'x': 0.07067751248062448, 'y': 1.4424042554392127}, {'x': 0.1581821251140786, 'y': 1.4039505211339234}]

Scores: [-0.023595376669542353, 0.8048257865186552, 0.05294575000372527, 0.10941057509098914]

Next configurations: [{'x': 0.99999998, 'y': 1.99999998}, {'x': 0.99999998, 'y': 1.99999998}, {'x': 0.99999998, 'y': 1.99999998}, {'x': 0.99999998, 'y': 1.99999998}]
{'fraction_non_alphanumeric': 0.11157270029673591, 'fraction_numerical': 0.16913946587537093, 'mean_word_length': 5.936213991769547, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '415069', 'license': 'MIT', 'n_tokens_mistral': 1491, 'n_tokens_neox': 1142, 'n_words': 303, 'path': 'notebooks/python/04. Batching.ipynb', 'repo_name': 'MindFoundry/optaas-tutorials', 'size': '3370'}