<jupyter_start><jupyter_text>PreparingFirst, we define the space of hyper-parameters optimization and we optimize the model with the non-stochastic algorithm.<jupyter_code>space = {'window': hp.quniform('window', 1, 30, 1),
         'count_upd': hp.quniform('count_upd', 1, 20, 1),
         'factor_upd': hp.quniform('factor_upd', 1, 10, 0.1),
         'thresh_upd': hp.quniform('thresh_upd', 0, 10, 0.1)}

model = Nested(data_folder)

base_res = model.optimize(BFGS, **{'verbose': False, 'nbr_epochs': 50, 'thresh': 1e-6})

print("Number of epochs for non-stocha: {}".format(base_res.nep))<jupyter_output>Number of epochs for non-stocha: 15<jupyter_text>Objective functionWe define the objective function required for hyperopt.<jupyter_code>def objective(model, algo, params, base_val, data_folder):
    win = params['window']
    cnt_upd = params['count_upd']
    fct_upd = params['factor_upd']
    thrsh_upd = params['thresh_upd']
    
    epochs = []
    fval = []
    for i in range(5):
        model = Nested(data_folder)
        
        res = model.optimize(algo,
                             **{'verbose': False, 'batch': 100, 'nbr_epochs': 50, 'thresh': 1e-6,
                                'thresh_upd': thrsh_upd,
                                'factor_upd': fct_upd, 'count_upd': cnt_upd, 'window': win})
        epochs.append(res.nep)
        fval.append(res.fun)

    if np.abs(np.mean(fval) - base_val) < 1e-5:
        status = STATUS_OK
    else:
        status = STATUS_FAIL

    return {'loss': np.max(epochs), 'value': np.mean(fval), 'status': status, 'epochs': epochs}<jupyter_output><empty_output><jupyter_text>HyperoptWe use hyperopt to find the best hyperparameters for the ABS algorithm.<jupyter_code>trials = Trials()
best = fmin(fn=lambda x: objective(model, SBFGSABS, x, base_res.fun, data_folder),
            space=space,
            algo=tpe.suggest,
            max_evals=100,
            trials=trials)<jupyter_output>01-04-2019:16:48:43,281 INFO     [tpe.py:837] tpe_transform took 0.001128 seconds
01-04-2019:16:48:43,282 INFO     [tpe.py:867] TPE using 0 trials
01-04-2019:16:49:49,832 INFO     [tpe.py:837] tpe_transform took 0.001172 seconds
01-04-2019:16:49:49,833 INFO     [tpe.py:865] TPE using 1/1 trials with best loss 13.447991
01-04-2019:16:50:44,71 INFO     [tpe.py:837] tpe_transform took 0.004886 seconds
01-04-2019:16:50:44,72 INFO     [tpe.py:865] TPE using 2/2 trials with best loss 13.447991
01-04-2019:16:52:08,192 INFO     [tpe.py:837] tpe_transform took 0.002336 seconds
01-04-2019:16:52:08,194 INFO     [tpe.py:865] TPE using 3/3 trials with best loss 13.447991
01-04-2019:16:54:10,702 INFO     [tpe.py:837] tpe_transform took 0.001435 seconds
01-04-2019:16:54:10,702 INFO     [tpe.py:865] TPE using 4/4 trials with best loss 13.447991
01-04-2019:16:55:08,730 INFO     [tpe.py:837] tpe_transform took 0.001241 seconds
01-04-2019:16:55:08,730 INFO     [tpe.py:865] TPE using 5/5 trials with best [...]<jupyter_text>After the optimization failedWe reload the model and try to run it again.<jupyter_code>model = Nested(data_folder)

base_res = model.optimize(BFGS, **{'verbose': False, 'nbr_epochs': 50, 'thresh': 1e-6})<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13697347893915757, 'fraction_numerical': 0.11357254290171608, 'mean_word_length': 3.547517730496454, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '61019', 'license': 'MIT', 'n_tokens_mistral': 1450, 'n_tokens_neox': 1214, 'n_words': 272, 'path': 'code/notebooks/[FAILING]Nested_BFGS_hyperopt.ipynb', 'repo_name': 'glederrey/biogeme_hp_nested', 'size': '3205'}