<jupyter_start><jupyter_text>Most of the time, we want our optimizer to quit when it converges or when the parameter space has been fully explored. Other times, we'd rather optimize continuously, keeping the coordinates near a (potentially drifting) maximum. In these cases, we can pass the "continuous=True" flag into certain optimizers. This overrides a specified number of iterations and will instead run the optimization algorithm until interrupted.Let's see an example of continuous optimization. We define a Gaussian objective function with a second (uncontrolled) parameter determining the location of the peak. We'll start continuous optimization and watch how the GradientDescent optimizer responds when we change the peak location:<jupyter_code>import numpy as np
from parametric import Parameter
from optimistic import experiment, GridSearch, GradientDescent
import time 

x = Parameter('x', 0.5)
x0 = Parameter('x0', 0)

@experiment
def gaussian():
    time.sleep(0.0005)
    return np.exp(-(x-x0)**2)

gd = GradientDescent(gaussian, 
                     show_progress=False, 
                     record_data=False, 
                     display=True, 
                     continuous=True, 
                     learning_rate=8e-2,
                     threaded=True).add_parameter(x, bounds=(-10, 10))
gd.run()

x0(0.2)<jupyter_output><empty_output><jupyter_text>Behind the scenesTo streamline the code, we'd like to choose between "while True" and "for i in range(iterations)" statements depending on the truth value of "continuous." This can be done very cleanly using a generator overriding range:<jupyter_code>def custom_range(iterations, continuous=False):
    if not continuous:
        yield from range(iterations)
    else:
        i = 0
        while True:
            yield i 
            i = (i+1) % iterations
  
for i in custom_range(10, continuous=False):
    print(i)<jupyter_output><empty_output><jupyter_text>If continuous==False, then this behaves exactly like a list. However, if continuous==True, then the generator will repeatedly reset i to 0 and repeat the incrementing until the execution is interrupted. This function is defined in the Algorithm base class, so algorithms using "for i in range(iterations)" will automatically run forever if the "continuous" flag is set to True. We also have an "iterate" function which replaces the default Python behavior of "for x in X", where X is a list. If self.continuous is True, the list will be iterated through repeatedly. This function looks like this:<jupyter_code>def iterate(lst, continuous=False):
    if not continuous:
        yield from list(lst)
    else:
        i = 0
        while True:
            yield lst[i]
            i = (i+1) % len(lst)
            
            
points = [0, 2, 3, 5]

for p in iterate(points, continuous=False):
    print(p)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07125477928397636, 'fraction_numerical': 0.011122697254084116, 'mean_word_length': 3.3871951219512195, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '473818', 'license': 'MIT', 'n_tokens_mistral': 798, 'n_tokens_neox': 741, 'n_words': 327, 'path': 'tutorials/Continuous optimization.ipynb', 'repo_name': 'lab-api/optimistic', 'size': '2877'}