<jupyter_start><jupyter_text>1.Try a Support Vector Machine regressor (`sklearn.svm.SVR`), with various hyperparameters such as `kernel="linear"` (with various values for the `C` hyperparameter) or `kernel="rbf"` (with various values for the `C` and `gamma` hyperparameters). Don’t worry about what these hyperparameters mean for now. How does the best `SVR` predictor perform?<jupyter_code>from sklearn.svm import SVR

# default SVM with linear kernel
svm_reg = SVR(kernel="linear")
svm_reg.fit(housing_prepared, housing_labels)
housing_predictions = svm_reg.predict(housing_prepared)
svm_mse = mean_squared_error(housing_labels, housing_predictions)
svm_rmse = np.sqrt(svm_mse)
svm_rmse
param_grid = [
    {'kernel': ['linear'], 'C': [1, 3, 10, 30, 100, 300, 1000, 3000, 10000, 30000]}
]

svm_reg = SVR()
grid_search = GridSearchCV(svm_reg, param_grid, cv=5, scoring='neg_mean_squared_error')
grid_search.fit(housing_prepared, housing_labels)
grid_search.best_params_
np.sqrt(-grid_search.best_score_)
param_grid = [
    {'kernel': ['rbf'], 'C': [10, 30, 100, 300, 1000, 3000, 10000, 30000],
    'gamma': [0.01, 0.03, 0.1, 0.3, 1, 3]}
]

svm_reg = SVR()
grid_search = GridSearchCV(svm_reg, param_grid, cv=5, scoring='neg_mean_squared_error')
grid_search.fit(housing_prepared, housing_labels)
feature_importances = grid_search.best_estimator_.feature_importances_<jupyter_output><empty_output><jupyter_text>2. Try replacing `GridSearchCV` with `RandomizedSearchCV`.<jupyter_code># replacing in the context code
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint

param_distribs = {
    'n_estimators': randint(low=1, high=100),
    'max_features': randint(low=1, high=8)
}

forest_reg = RandomForestRegressor()
rnd_search = RandomizedSearchCV(forest_reg, param_distributions=param_distribs,
                                n_iter=10, cv=5, scoring='neg_mean_squared_error', random_state=42)
rnd_search.fit(housing_prepared, housing_labels)
rnd_search.best_params_
np.sqrt(-rnd_search.best_score_)
# the following code is referring to the author's solution
# https://github.com/ageron/handson-ml/blob/master/02_end_to_end_machine_learning_project.ipynb

from scipy.stats import expon, reciprocal

param_distribs = {
    'kernel':['linear', 'rbf'],
    'C': reciprocal(20, 200000),
    'gamma': expon(scale=1.0)
}

svm_reg = SVR()
rnd_search = RandomizedSearchCV(svm_reg, param_distributions=param_distribs,
                                n_iter=50, cv=5, scoring='neg_mean_squared_error',
                                verbose=2, n_jobs=4, random_state=42)
rnd_search.fit(housing_prepared, housing_labels)
np.sqrt(-rnd_search.best_score_)
rnd_search.best_params_<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11502516175413371, 'fraction_numerical': 0.037383177570093455, 'mean_word_length': 5.722222222222222, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '501653', 'license': 'MIT', 'n_tokens_mistral': 1084, 'n_tokens_neox': 973, 'n_words': 202, 'path': 'Chapter 2/solution.ipynb', 'repo_name': 'HaelChan/Hands-on_ML', 'size': '2782'}