<jupyter_start><jupyter_text>Ray Tune Basics* https://docs.ray.io/en/latest/tune/index.html* https://docs.ray.io/en/latest/tune/examples/tune_basic_example.html* **NOTE:**    * **This is search space based optimization such as that used for hyperparameter search problems.**    * **This is not constratint based optimization.**<jupyter_code>import random
import numpy as np
import pandas as pd
import ray
from ray import tune<jupyter_output><empty_output><jupyter_text>Minimize Objective  Step 1: Define Objective Function<jupyter_code>def objective(x, intercept=4):
    """Find the MIN of this parabola"""
    return (x - intercept)**2 + random.random()<jupyter_output><empty_output><jupyter_text>Step 2: Visualize Objective Function (if possible)<jupyter_code>sweep = pd.DataFrame(np.array([objective(i) for i in np.arange(0, 8)]))
sweep.plot()<jupyter_output><empty_output><jupyter_text>Step 3: Optimize to find Maxima<jupyter_code>def training_function(config):
    # Hyperparameters
    x = config["x"]
    # Iterative training function - can be any arbitrary training procedure.
    intermediate_score = objective(x)
    # Feed the score back back to Tune.
    tune.report(loss=intermediate_score)
help(tune.run)
analysis = tune.run(
    training_function,
    metric = "loss",
    mode = "min",
    name="minimize_experiment",
    config={
        "x": tune.uniform(0,8)        
    },
    num_samples=50 
)<jupyter_output>2022-01-15 15:47:29,852	WARNING function_runner.py:561 -- Function checkpointing is disabled. This may result in unexpected behavior when using checkpointing features or certain schedulers. To enable, set the train function arguments to be `func(config, checkpoint_dir=None)`.
2022-01-15 15:47:29,977	WARNING tune.py:570 -- Tune detects GPUs, but no trials are using GPUs. To enable trials to use GPUs, set tune.run(resources_per_trial={'gpu': 1}...) which allows Tune to expose 1 GPU to each trial. You can also override `Trainable.default_resource_request` if using the Trainable API.<jupyter_text>What do these messahes mean?https://docs.ray.io/en/latest/tune/tutorials/tune-lifecycle.htmllifecycle-of-a-trial  Step 4: Results<jupyter_code>print("Best config: ", analysis.get_best_config(metric="loss", mode="min"))
# Get a dataframe for analyzing trial results.
analysis.results_df<jupyter_output>C:\Users\Nikhil\.conda\envs\ray-test\lib\site-packages\ray\tune\analysis\experiment_analysis.py:262: UserWarning: Dataframes will use '/' instead of '.' to delimit nested result keys in future versions of Ray. For forward compatibility, set the environment variable TUNE_RESULT_DELIM='/'
  warnings.warn(<jupyter_text>Maximum Objective  Step 1: Define Objective Function<jupyter_code>def objective(x, intercept=4):
    """Find the MAX of this parabola"""
    return 16 - (x - intercept)**2 + random.random()<jupyter_output><empty_output><jupyter_text>Step 2: Visualize Objective Function (if possible)<jupyter_code>sweep = pd.DataFrame(np.array([objective(i) for i in np.arange(0, 8)]))
sweep.plot()<jupyter_output><empty_output><jupyter_text>Step 3: Optimize to find Maxima<jupyter_code>def training_function(config):
    # Hyperparameters
    x = config["x"]
    # Iterative training function - can be any arbitrary training procedure.
    intermediate_score = objective(x)
    # Feed the score back back to Tune.
    tune.report(loss=intermediate_score)
analysis = tune.run(
    training_function,
    metric = "loss",
    mode = "max",
    name="maximize_experiment",
    config={
        "x": tune.uniform(0,8)        
    },
    num_samples=50 
)<jupyter_output>2022-01-15 15:48:29,374	WARNING tune.py:570 -- Tune detects GPUs, but no trials are using GPUs. To enable trials to use GPUs, set tune.run(resources_per_trial={'gpu': 1}...) which allows Tune to expose 1 GPU to each trial. You can also override `Trainable.default_resource_request` if using the Trainable API.<jupyter_text>Step 4: Results<jupyter_code>print("Best config: ", analysis.get_best_config(metric="loss", mode="max"))
# Get a dataframe for analyzing trial results.
analysis.results_df<jupyter_output><empty_output><jupyter_text>Shutdown<jupyter_code>#####################################
#### Dont forget to shutdown ray ####
#####################################
ray.shutdown()
assert ray.is_initialized() is False<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12884703720716584, 'fraction_numerical': 0.021359669269637116, 'mean_word_length': 5.257183908045977, 'pattern_counts': {'":': 2, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '224467', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1465, 'n_tokens_neox': 1369, 'n_words': 367, 'path': 'analysis/ray_tune_basic_optimization.ipynb', 'repo_name': 'ngupta23/ray-test', 'size': '4354'}