<jupyter_start><jupyter_text>Ordinary Least Squares and Ridge Regression VarianceDue to the few points in each dimension and the straightline that linear regression uses to follow these pointsas well as it can, noise on the observations will causegreat variance as shown in the first plot. Every line's slopecan vary quite a bit for each prediction due to the noiseinduced in the observations.Ridge regression is basically minimizing a penalised versionof the least-squared function. The penalising `shrinks` thevalue of the regression coefficients.Despite the few data points in each dimension, the slopeof the prediction is much more stable and the variancein the line itself is greatly reduced, in comparison to thatof the standard linear regression<jupyter_code>print(__doc__)


# Code source: <NAME>
# Modified for documentation by <NAME>
# License: BSD 3 clause


import numpy as np
import matplotlib.pyplot as plt

from sklearn import linear_model

X_train = np.c_[.5, 1].T
y_train = [.5, 1]
X_test = np.c_[0, 2].T

np.random.seed(0)

classifiers = dict(ols=linear_model.LinearRegression(),
                   ridge=linear_model.Ridge(alpha=.1))

fignum = 1
for name, clf in classifiers.items():
    fig = plt.figure(fignum, figsize=(4, 3))
    plt.clf()
    plt.title(name)
    ax = plt.axes([.12, .12, .8, .8])

    for _ in range(6):
        this_X = .1 * np.random.normal(size=(2, 1)) + X_train
        clf.fit(this_X, y_train)

        ax.plot(X_test, clf.predict(X_test), color='.5')
        ax.scatter(this_X, y_train, s=3, c='.5', marker='o', zorder=10)

    clf.fit(X_train, y_train)
    ax.plot(X_test, clf.predict(X_test), linewidth=2, color='blue')
    ax.scatter(X_train, y_train, s=30, c='r', marker='+', zorder=10)

    ax.set_xticks(())
    ax.set_yticks(())
    ax.set_ylim((0, 1.6))
    ax.set_xlabel('X')
    ax.set_ylabel('y')
    ax.set_xlim(0, 2)
    fignum += 1

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11433005690636316, 'fraction_numerical': 0.01965856182100362, 'mean_word_length': 3.921119592875318, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '618025', 'license': 'MIT', 'n_tokens_mistral': 696, 'n_tokens_neox': 656, 'n_words': 206, 'path': 'scikit-learn-official-examples/linear_model/plot_ols_ridge_variance.ipynb', 'repo_name': 'gopala-kr/ds-notebooks', 'size': '1949'}