<jupyter_start><jupyter_text>Swap test and train<jupyter_code>x_train, x_test = x_test, x_train
y_train, y_test = y_test, y_train<jupyter_output><empty_output><jupyter_text>Now the same code as before, but mse_2<jupyter_code>reg_model = LinearRegression()
reg_model.fit(x_train, y_train)
y_pred_test = reg_model.predict(x_test)
test_mse2 = mean_squared_error(y_test, y_pred_test)
test_mse2
mse_2fold = np.mean([test_mse1, test_mse2])
mse_2fold<jupyter_output><empty_output><jupyter_text>Now the same, but using the special library<jupyter_code>from sklearn.model_selection import KFold, cross_val_score
kfold = KFold(n_splits = 2)
reg_model = LinearRegression()
mse_array = cross_val_score(estimator=reg_model, X=x, y=y, cv=kfold, scoring='neg_mean_squared_error')
mse_array

-np.mean(mse_array)
import sklearn
# Look at the possible metrics and find the one corresponding to MSE:
print(sorted(sklearn.metrics.SCORERS.keys()))
from sklearn.model_selection import RepeatedKFold
rkf = RepeatedKFold(n_splits=2, n_repeats=2)
reg_model = LinearRegression()
mse_array = cross_val_score(estimator=reg_model, X=x, y=y, cv=rkf, scoring='neg_mean_squared_error')
mse_array, -np.mean(mse_array)<jupyter_output><empty_output><jupyter_text>Leave one out (LOO). If by hand, we can do 4 fits, take their 4 MSE's and average them up. But it takes space. We do it just by using the lbrary this time.<jupyter_code>from sklearn.model_selection import LeaveOneOut
loo = LeaveOneOut()
reg_model = LinearRegression()
mse_array = cross_val_score(estimator=reg_model, X=x, y=y, cv=loo, scoring='neg_mean_squared_error')
mse_array, -np.mean(mse_array)<jupyter_output><empty_output><jupyter_text>You can also do the same with the following classes:<jupyter_code>from sklearn.model_selection import LeavePOut
from sklearn.model_selection import ShuffleSplit
from sklearn.model_selection import StratifiedKFold<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0930475692629378, 'fraction_numerical': 0.006272869837950863, 'mean_word_length': 8.815384615384616, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '386385', 'license': 'MIT', 'n_tokens_mistral': 743, 'n_tokens_neox': 719, 'n_words': 155, 'path': 'Chapter 5/Python/cross-validation/1. simple_CV.ipynb', 'repo_name': 'borisgarbuzov/schulich_data_science_1', 'size': '1913'}