<jupyter_start><jupyter_text>Learning curveIn the following we would like to estimate if more training data really will improve the score of our models. We exemplify this using a simple classification data set with information on success/failure per data point. We use the sklearn function learning_curve to visualize what happens if we assign different ratios to training/test data sets and briefly compare the results for KNN and logictic regression learning algorithms.<jupyter_code>#import necessary packages
import pandas as pd

import numpy as np

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

%matplotlib inline
import matplotlib.pyplot as plt

from helper import plot_classifier #helper.py is saved in the repository
#define data frame
df = pd.read_csv("classification.csv")

df.head()
df.shape
#define variables
X = df[["age","interest"]].values

Y = df["success"].values # single brackets


#rescale data set
scaler = StandardScaler()
scaler.fit(X)

X = scaler.transform(X)
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import learning_curve

#automatically separates data set into different ratios of training/test sets
train_sizes_abs, train_scores, test_scores = learning_curve(KNeighborsClassifier(), X, Y)
#?learning_curve
#plotting the results for averaged scores over training set sizes
plt.plot(train_sizes_abs, np.mean(train_scores, axis = 1))
plt.plot(train_sizes_abs, np.mean(test_scores, axis = 1))
plt.show()<jupyter_output><empty_output><jupyter_text>Comment: The blue curve indicates the training score data. Even with much more data an improvement may not be reached. Notice that the data set has < 300 entries. We may compare with a classification via logistic regression:<jupyter_code>from sklearn.linear_model import LogisticRegression
from sklearn.utils import shuffle

#good if one does not know if data is ordered
X, Y = shuffle(X,Y)

#one may also use cross-validation by settin cv = integer
train_sizes_abs, train_scores, test_scores = learning_curve(LogisticRegression(), X, Y, cv = 10)
#plotting the results for averaged scores over training set sizes
plt.plot(train_sizes_abs, np.mean(train_scores, axis = 1))
plt.plot(train_sizes_abs, np.mean(test_scores, axis = 1))
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06806506849315068, 'fraction_numerical': 0.003852739726027397, 'mean_word_length': 6.190769230769231, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '143853', 'license': 'MIT', 'n_tokens_mistral': 708, 'n_tokens_neox': 659, 'n_words': 261, 'path': 'KNNandlearningcurve.ipynb', 'repo_name': 'andreaspts/ML_KNN_Analyses_for_Classification', 'size': '2336'}