<jupyter_start><jupyter_text>Format DataFrame<jupyter_code>import pandas as pd
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
train_df = pd.DataFrame(data.data, columns=data.feature_names)
train_df["diagnosis"] = data.target

print(train_df.shape)
train_df.head()<jupyter_output>(569, 31)<jupyter_text>Set Up Environment<jupyter_code>from hyperparameter_hunter import Environment, CVExperiment

env = Environment(
    train_dataset=train_df,
    results_path="HyperparameterHunterAssets",
    target_column="diagnosis",
    metrics=["roc_auc_score"],
    cv_type="StratifiedKFold",
    cv_params=dict(n_splits=5, random_state=32),
    verbose=1,
)<jupyter_output>Cross-Experiment Key:   'aZlwCukh8RVC3A3BIv1DK27sZYyUCsegMFpr2R4JB60='<jupyter_text>We're initializing our `Environment` with `verbose=1` to tell our experiments to only log the essentials because we're about to run lots of experiments.Now that HyperparameterHunter has an active `Environment`, we can do two things: 1. Perform Experiments<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC, LinearSVC, NuSVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier, GradientBoostingClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis, QuadraticDiscriminantAnalysis
from sklearn.neural_network import MLPClassifier

experiment_0 = CVExperiment(KNeighborsClassifier, {})
experiment_1 = CVExperiment(SVC, {})
experiment_2 = CVExperiment(LinearSVC, {})
experiment_3 = CVExperiment(NuSVC, {})
experiment_4 = CVExperiment(DecisionTreeClassifier, {})
experiment_5 = CVExperiment(RandomForestClassifier, {})
experiment_6 = CVExperiment(AdaBoostClassifier, {})
experiment_7 = CVExperiment(GradientBoostingClassifier, {})
experiment_8 = CVExperiment(GaussianNB, {})
experiment_9 = CVExperiment(LinearDiscriminantAnalysis, {})
experiment_10 = CVExperiment(QuadraticDiscriminantAnalysis, {})
experiment_11 = CVExperiment(MLPClassifier, {})<jupyter_output><21:06:45> Validated Environment:  'aZlwCukh8RVC3A3BIv1DK27sZYyUCsegMFpr2R4JB60='
<21:06:45> Initialized Experiment: 'b41a3b1c-ef03-47b5-82f1-00cf74b43403'
<21:06:45> Hyperparameter Key:     '<KEY>
<21:06:45> 
<21:06:46> 
<21:06:46> FINAL:    OOF(roc_auc_score=0.92003)  |  Time Elapsed: 0.82798 s
<21:06:46> 
<21:06:46> Saving results for Experiment: 'b41a3b1c-ef03-47b5-82f1-00cf74b43403'<jupyter_text>Of course, SKLearn has many more algorithms than those shown here, but I think you get the idea.Notice that in all the above experiments, we gave `CVExperiment` `model_init_params={}`. Passing an empty dict tells it to use the default hyperparameters for the `model_initializer`, which it'll figure out on its own. 2. Hyperparameter OptimizationWe're just going to do optimization on one of the algorithms used above (`AdaBoostClassifier`); although, HyperparameterHunter can certainly do consecutive optimization rounds.Notice below that `optimizer` correctly identifies `experiment_6` as being the only saved experiment it can learn from because it's optimizing `AdaBoostClassifier`.<jupyter_code>from hyperparameter_hunter import RandomForestOptimization, Real, Integer, Categorical

optimizer = RandomForestOptimization(iterations=12, random_state=42)

optimizer.set_experiment_guidelines(
    model_initializer=AdaBoostClassifier,
    model_init_params=dict(
        n_estimators=Integer(25, 100),
        learning_rate=Real(0.5, 1.0),
        algorithm=Categorical(["SAMME", "SAMME.R"]),
    ),
)

optimizer.go()<jupyter_output>Validated Environment with key: "<KEY>
[31mSaved Result Files[0m
[31m_________________________________________________________________________________________[0m
 Step |       ID |   Time |      Value |   algorithm |   learning_rate |   n_estimators | 
Experiments matching cross-experiment key/algorithm: 1
Experiments fitting in the given space: 1
Experiments matching current guidelines: 1
    0 | 35a23c9d | 00m00s | [35m   0.96425[0m | [32m    SAMME.R[0m | [32m         1.0000[0m | [32m            50[0m | 
[31mHyperparameter Optimization[0m
[31m_________________________________________________________________________________________[0m
 Step |       ID |   Time |      Value |   algorithm |   learning_rate |   n_estimators | 
    1 | 7838c551 | 00m00s |    0.96425 |     SAMME.R |          0.6864 |             68 | 
    2 | e3f8b127 | 00m00s |    0.95438 |     SAMME.R |          0.8683 |             44 | 
    3 | ec5b82e8 | 00m00s |[...]
{'fraction_non_alphanumeric': 0.0914409534127844, 'fraction_numerical': 0.05828819068255688, 'mean_word_length': 5.7190684133915575, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '162119', 'license': 'MIT', 'n_tokens_mistral': 1700, 'n_tokens_neox': 1513, 'n_words': 333, 'path': 'examples/sklearn_examples/classification.ipynb', 'repo_name': 'mdjabc/hyperparameter_hunter', 'size': '4695'}