<jupyter_start><jupyter_text>K-means clusteringIt should not surprise you at this point to learn that doing unsupervised learning in scikit-learn is pretty easy (which is not to say that doing it *well* is easy, of course). The code looks almost exactly the same is it does in the unsupervised setting. The main difference is that unsupervised learning estimators take only `X` data—there is, by definition, no set of labels `y`.Let's cluster the observations in the above plot using the [k-means clustering](https://en.wikipedia.org/wiki/K-means_clustering) algorithm:<jupyter_code>from sklearn.cluster import KMeans

kmc = KMeans(3)
kmc.fit(X)

colors = kmc.predict(X)
plt.scatter(X[:, 0], X[:, 1], c=colors);<jupyter_output><empty_output><jupyter_text>Job well done?At this point we've developed fully operational machine learning workflows for three separate kinds of machine learning problems: regression, classification, and clustering. Relatively little work was involved in each case. In the case of the regression problem, for example, we initialized a linear regression model, fit it to some data, used it to generate predictions, and scored those predictions— all in 3 or 4 lines of code! This seems pretty great. Maybe we should just stop here, pat ourselves on the back for a job well done, and head home for the day.For reasons that will shortly become clear, though, calling it quits here would be a really bad idea.Let's dig a little deeper. We'll start by asking an important question that echoes back to the definition of machine learning as the study of systems that can improve their output by learning from experience. Specifically: how does our model's performance evolve as we give it more data?Think back to our (continuous) age prediction problem. Intuitively, we might expect that our $R^2$ will go up as we increase the size of our dataset (because having more data to learn from seems like it should be a good thing). But we should probably verify that.Instead of fitting our `LinearRegression` estimator to just one dataset, let's systematically vary our sample size over a large range, and fit a linear regression to each one (actually, to stabilize our performance estimates, we'll average over multiple permutations at each sample size).The code below is much more involved than it needs to be; as we'll see in the next section, scikit-learn includes a number of utilities that can achieve the same goal much more compactly and efficiently. But I think it can be helpful to explicitly lay out all of the steps we're going through before we replace them with a single line of black magic.<jupyter_code># initialize the OLS estimator
est = LinearRegression()

# we'll plot a separate panel for each feature set
feature_sets = ['domains', 'facets', 'items']

# evaluate performance at each of these sample sizes
sample_sizes = [100, 200, 500, 1000, 2000, 5000, 10000, 20000, 50000]

# number of permutations to average over at each sample size
n_reps = 10

# store results for all permutations, sample sizes, and feature sets
results = np.zeros((n_reps, len(sample_sizes), len(feature_sets)))

# loop over permutations
for i in range(n_reps):
    # loop over sample sizes
    for j, n in enumerate(sample_sizes):
        # get the appropriate features and labels
        *Xs, age = get_features(data, *feature_sets, 'AGE', n=n)
        # loop over feature sets
        for k, X in enumerate(Xs):
            # fit the model
            est.fit(X, age)
            # generate predictions
            pred_y = est.predict(X)
            # save R^2 in our results array
            results[i, j, k] = r2_score(age, pred_y)

# Compute means and stdevs for error bars 
r2_mean = results.mean(0)
r2_std = results.std(0)<jupyter_output><empty_output><jupyter_text>Now we can plot the resulting $R^2$ values for each feature set as a function of sample size:<jupyter_code># used to display axis tick labels on a linear scale
from matplotlib.ticker import ScalarFormatter

# Set up plots=
fig, axes = plt.subplots(1, 3, figsize=(15, 4), sharey=True)
plt.ylim(0, 1)

# Plot results
for i, label in enumerate(feature_sets):
    mean, sd = r2_mean[:, i], r2_std[:, i]
    ax = axes[i]
    line = ax.plot(sample_sizes, mean, 'o-')
    ax.set_xscale('log')
    ax.xaxis.set_major_formatter(ScalarFormatter())
    ax.set_xlabel("Sample size (n)", fontsize=14)
    ax.fill_between(sample_sizes, mean-sd, mean+sd, alpha=0.2)
    ax.set_title(label, fontsize=18)

# Add y-axis labels on both sides
axes[0].set_ylabel("$R^2$", fontsize=16);
axes[-1].set_ylabel('$R^2$', fontsize=16)
axes[-1].yaxis.set_label_position("right")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07888531618435156, 'fraction_numerical': 0.01607717041800643, 'mean_word_length': 4.4192799070847855, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '351507', 'license': 'MIT', 'n_tokens_mistral': 1414, 'n_tokens_neox': 1304, 'n_words': 618, 'path': 'content/courses/ml_intro/10_k_means_clustering/01_unsupervised_learning.ipynb', 'repo_name': 'greydongilmore/personal_website', 'size': '4665'}