<jupyter_start><jupyter_text>🏁 Wrap-up quiz**This quiz requires some programming to be answered.** Load the dataset file named `penguins.csv` with the following command:<jupyter_code>import pandas as pd


penguins = pd.read_csv("../datasets/penguins.csv")

columns = ["Body Mass (g)", "Flipper Length (mm)", "Culmen Length (mm)"]
target_name = "Species"

# Remove lines with missing values for the columns of interestes
penguins_non_missing = penguins[columns + [target_name]].dropna()

data = penguins_non_missing[columns]
target = penguins_non_missing[target_name]<jupyter_output><empty_output><jupyter_text>`penguins` is a pandas dataframe. The column "Species" contains the targetvariable. We extract through numerical columns that quantify various attributesof animals and our goal is try to predict the species of the animal based onthose attributes stored in the dataframe named `data`.Inspect the loaded data to select the correct assertions:  Question 1Select the correct assertions from the following proposals.- a) The problem to be solved is a regression problem- b) The problem to be solved is a binary classification problem  (exactly 2 possible classes)- c) The problem to be solved is a multiclass classification problem  (more than 2 possible classes)- d) The proportions of the class counts are balanced: there are approximately  the same number of rows for each class- e) The proportions of the class counts are imbalanced: some classes have more  than twice as many rows than others)- f) The input features have similar dynamic ranges (or scales)_Select several answers_Hint: `data.describe()`, and `target.value_counts()` are methodsthat are helpful to answer to this question.<jupyter_code>data.describe()
target.value_counts()<jupyter_output><empty_output><jupyter_text>+++Let's now consider the following pipeline:<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline


model = Pipeline(steps=[
    ("preprocessor", StandardScaler()),
    ("classifier", KNeighborsClassifier(n_neighbors=5)),
])<jupyter_output><empty_output><jupyter_text>Question 2Evaluate the pipeline using 10-fold cross-validation using the`balanced-accuracy` scoring metric to choose the correct statements.Use `sklearn.model_selection.cross_validate` with`scoring="balanced_accuracy"`.Use `model.get_params()` to list the parameters of the pipeline and use`model.set_params(param_name=param_value)` to update them.- a) The average cross-validated test `balanced_accuracy` of the above pipeline is between 0.9 and 1.0- b) The average cross-validated test `balanced_accuracy` of the above pipeline is between 0.8 and 0.9- c) The average cross-validated test `balanced_accuracy` of the above pipeline is between 0.5 and 0.8- d) Using `n_neighbors=5` is much better than `n_neighbors=51`- e) Preprocessing with `StandardScaler` is much better than using the     raw features (with `n_neighbors=5`)_Select several answers_<jupyter_code>from sklearn.model_selection import cross_validate, ShuffleSplit

model.set_params(classifier__n_neighbors=5)
cv = ShuffleSplit(random_state=0)
cv_results = cross_validate(model, data, target, return_train_score=True,
                            cv=cv, scoring='balanced_accuracy')
cv_results = pd.DataFrame(cv_results)
print(f'model {model.get_params()}')
print(f"{cv_results.mean()}\n")

model.set_params(classifier__n_neighbors=51)
cv_results = cross_validate(model, data, target, return_train_score=True,
                            cv=cv, scoring='balanced_accuracy')
cv_results = pd.DataFrame(cv_results)
print(f'model {model.get_params()}')
print(f"{cv_results.mean()}\n")

model.set_params(classifier__n_neighbors=5)
model.set_params(preprocessor__with_mean=False, preprocessor__with_std=False)
cv_results = cross_validate(model, data, target, return_train_score=True,
                            cv=cv, scoring='balanced_accuracy')
cv_results = pd.DataFrame(cv_results)
print(f'model {model.get_params()}')
print(f"{cv_results.mean()}\n")<jupyter_output>model {'memory': None, 'steps': [('preprocessor', StandardScaler()), ('classifier', KNeighborsClassifier())], 'verbose': False, 'preprocessor': StandardScaler(), 'classifier': KNeighborsClassifier(), 'preprocessor__copy': True, 'preprocessor__with_mean': True, 'preprocessor__with_std': True, 'classifier__algorithm': 'auto', 'classifier__leaf_size': 30, 'classifier__metric': 'minkowski', 'classifier__metric_params': None, 'classifier__n_jobs': None, 'classifier__n_neighbors': 5, 'classifier__p': 2, 'classifier__weights': 'uniform'}
fit_time       0.002935
score_time     0.002327
test_score     0.954629
train_score    0.972513
dtype: float64

model {'memory': None, 'steps': [('preprocessor', StandardScaler()), ('classifier', KNeighborsClassifier(n_neighbors=51))], 'verbose': False, 'preprocessor': StandardScaler(), 'classifier': KNeighborsClassifier(n_neighbors=51), 'preprocessor__copy': True, 'preprocessor__with_mean': True, 'preprocessor__with_std': True, 'classifier__algorithm': 'auto[...]<jupyter_text>+++We will now study the impact of different preprocessors defined in the list below:<jupyter_code>from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import QuantileTransformer
from sklearn.preprocessing import PowerTransformer


all_preprocessors = [
    None,
    StandardScaler(),
    MinMaxScaler(),
    QuantileTransformer(n_quantiles=100),
    PowerTransformer(method="box-cox"),
]<jupyter_output><empty_output><jupyter_text>The [Box-Coxmethod](https://en.wikipedia.org/wiki/Power_transformBox%E2%80%93Cox_transformation)is common preprocessing strategy for positive values. The other preprocessorswork both for any kind of numerical features. If you are curious to read thedetails about those method, please feel free to read them up in the[preprocessingchapter](https://scikit-learn.org/stable/modules/preprocessing.html) of thescikit-learn user guide but this is not required to answer the quiz questions.  Question 3Use `sklearn.model_selection.GridSearchCV` to study the impact of the choice ofthe preprocessor and the number of neighbors on the 10-fold cross-validated`balanced_accuracy` metric. We want to study the `n_neighbors` in the range`[5, 51, 101]` and `preprocessor` in the range `all_preprocessors`.Let us consider that a model is significantly better than another if the itsmean test score is better than the mean test score of the alternative by morethan the standard deviation of its test score.Which of the following statements hold:- a) The best model with `StandardScaler` is significantly better than using any other processor- b) Using any of the preprocessors is always better than using no processor, irrespective of the value `of n_neighbors`- c) The model with `n_neighbors=5` and `StandardScaler` is significantly better than the model with `n_neighbors=51` and `StandardScaler`.- d) The model with `n_neighbors=51` and `StandardScaler` is significantly better than the model with `n_neighbors=101` and `StandardScaler`.Hint: pass `{"preprocessor": all_preprocessors, "classifier__n_neighbors": [5, 51, 101]}` for the `param_grid` argument to the `GridSearchCV` class._Select several answers_<jupyter_code>from sklearn.model_selection import cross_validate
from sklearn.model_selection import GridSearchCV

param_grid = {"preprocessor": all_preprocessors, "classifier__n_neighbors": [5, 51, 101]}
model_grid_search = GridSearchCV(model, param_grid=param_grid,
                                 n_jobs=-1, cv=10, scoring='balanced_accuracy')

model_grid_search.fit(data, target)
cv_results = pd.DataFrame(model_grid_search.cv_results_).sort_values(
    "mean_test_score", ascending=False)
cv_results.head()
column_results = [f"param_{name}" for name in param_grid.keys()]
column_results += [
    "mean_test_score", "std_test_score", "rank_test_score"]
cv_results = cv_results[column_results]
cv_results<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09542271135567784, 'fraction_numerical': 0.013256628314157078, 'mean_word_length': 6.4114921223354955, 'pattern_counts': {'":': 4, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '268744', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2459, 'n_tokens_neox': 2332, 'n_words': 737, 'path': 'jupyter-book/tuning/tuning_questions.ipynb', 'repo_name': 'castorfou/scikit-learn-mooc', 'size': '7996'}