<jupyter_start><jupyter_text>📝 Exercise M3.02The goal is to find the best set of hyperparameters which maximize thestatistical performance on a training set.Here again with limit the size of the training set to make computationrun faster. Feel free to increase the `train_size` value if your computeris powerful enough.<jupyter_code>import numpy as np
import pandas as pd

adult_census = pd.read_csv("../datasets/adult-census.csv")

target_name = "class"
target = adult_census[target_name]
data = adult_census.drop(columns=[target_name, "education-num"])

from sklearn.model_selection import train_test_split

data_train, data_test, target_train, target_test = train_test_split(
    data, target, random_state=42)<jupyter_output><empty_output><jupyter_text>Create your machine learning pipelineYou should:* preprocess the categorical columns using a `OneHotEncoder` and use a  `StandardScaler` to normalize the numerical data.* use a `LogisticRegression` as a predictive model. Start by defining the columns and the preprocessing pipelines to be appliedon each columns.<jupyter_code>from sklearn.preprocessing import OneHotEncoder
from sklearn.preprocessing import StandardScaler

categorical_preprocessor = OneHotEncoder()
numerical_preprocessor = StandardScaler()<jupyter_output><empty_output><jupyter_text>Subsequently, create a `ColumnTransformer` to redirect the specific columnsa preprocessing pipeline.<jupyter_code>from sklearn.compose import ColumnTransformer
from sklearn.compose import make_column_selector as selector

preprocessor = ColumnTransformer([
    ('cat-preprocessor', categorical_preprocessor, selector(dtype_include=object)),
    ('num-preprocessor', numerical_preprocessor, selector(dtype_include='number'))],
    remainder='passthrough', sparse_threshold=0)<jupyter_output><empty_output><jupyter_text>Finally, concatenate the preprocessing pipeline with a logistic regression.<jupyter_code>from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression

model = make_pipeline(preprocessor, LogisticRegression())<jupyter_output><empty_output><jupyter_text>Use a `RandomizedSearchCV` to find the best set of hyperparameters by tuningthe following parameters of the `model`:- the parameter `C` of the `LogisticRegression` with values ranging from  0.001 to 10. You can use a log-uniform distribution  (i.e. `scipy.stats.loguniform`);- the parameter `with_mean` of the `StandardScaler` with possible values  `True` or `False`;- the parameter `with_std` of the `StandardScaler` with possible values  `True` or `False`.Once the computation has completed, print the best combination of parametersstored in the `best_params_` attribute.<jupyter_code>from sklearn import set_config
set_config(display='diagram')
model
model.get_params()
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import loguniform

# the parameter C of the LogisticRegression with values ranging from 0.001 to 10. You can use a log-uniform distribution (i.e. scipy.stats.loguniform);
# the parameter with_mean of the StandardScaler with possible values True or False;
# the parameter with_std of the StandardScaler with possible values True or False.
param_distributions = {
    "logisticregression__C": loguniform(0.001, 10),
    "columntransformer__num-preprocessor__with_mean": [True, False],
    "columntransformer__num-preprocessor__with_std": [True, False],
}

model_random_search = RandomizedSearchCV(
    model, param_distributions=param_distributions,
    n_iter=20, error_score=np.nan, n_jobs=-1, verbose=1)
model_random_search.fit(data_train, target_train)
model_random_search.best_params_<jupyter_output>Fitting 5 folds for each of 20 candidates, totalling 100 fits
{'fraction_non_alphanumeric': 0.0738850080601827, 'fraction_numerical': 0.009134873723804407, 'mean_word_length': 7.07592190889371, 'pattern_counts': {'":': 3, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '650745', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1090, 'n_tokens_neox': 1051, 'n_words': 338, 'path': 'notebooks/parameter_tuning_ex_03.ipynb', 'repo_name': 'castorfou/scikit-learn-mooc', 'size': '3722'}