<jupyter_start><jupyter_text>K-FoldCross-validation is a resampling procedure used to evaluate machine learning models on alimited data sample. The procedure has a single parameter called k that refers to the numberof groups that a given data sample is to be split into. As such, the procedure is often calledk-fold cross-validation. Variations on Cross-Validation*   **Stratified**: each fold has the same proportion of observations with a given categorical value.*   **LOOCV** (leave-one-out cross-validation): *k* may be set to the total number of observations in the dataset such that each observation is given a chance to be held out of the dataset.*   **Repeated**: This is where the *k-fold* cross-validation procedure is repeated *n* times, where importantly, the data sample is shuffled prior to each repetition, which results in a different split of the sample.<jupyter_code># scikit-learn k-fold cross-validation
from numpy import array
from sklearn.model_selection import KFold
# Data sample
data = array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6])

# Define cross-validation folds
kfold = KFold(n_splits=3, shuffle=True, random_state=1)

# Enumerate splits
for train, test in kfold.split(data):
  print('train: %s, test %s' % (data[train], data[test]))<jupyter_output>train: [0.1 0.4 0.5 0.6], test [0.2 0.3]
train: [0.2 0.3 0.4 0.6], test [0.1 0.5]
train: [0.1 0.2 0.3 0.5], test [0.4 0.6]<jupyter_text>Scikit-learn Example (Traditional Train-Test-Split)<jupyter_code>import numpy as np
from sklearn.model_selection import train_test_split
from sklearn import datasets
from sklearn import svm
X, y = datasets.load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)

print(X.shape, y.shape)
print(X_train.shape, y_train.shape)
print(X_test.shape, y_test.shape)
clf = svm.SVC(kernel='linear', C=1).fit(X_train, y_train)
clf.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Scikit-learn Example (Using K-Fold Cross-Validation)<jupyter_code>from sklearn.model_selection import cross_val_score
clf = svm.SVC(kernel='linear', C=1)
scores = cross_val_score(clf, X, y, cv=5)
scores
print('Accuracy: %0.2f (+/- %0.2f)' % (scores.mean(), scores.std() * 2))<jupyter_output>Accuracy: 0.98 (+/- 0.03)
{'fraction_non_alphanumeric': 0.11876379690949228, 'fraction_numerical': 0.029580573951434878, 'mean_word_length': 5.347338935574229, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '473561', 'license': 'Apache-2.0', 'n_tokens_mistral': 824, 'n_tokens_neox': 785, 'n_words': 255, 'path': 'Cross-Validation/K-fold.ipynb', 'repo_name': 'DouglasAltwig/Data-Science-Foundation', 'size': '2265'}