<jupyter_start><jupyter_text>Classic Supervised Regression using Stochastic Gradient Descent* More information: https://towardsdatascience.com/machine-learning-project-predicting-boston-house-prices-with-regression-b4e47493633d<jupyter_code>%matplotlib inline

import matplotlib.pyplot as plt
import sklearn
sklearn.__version__<jupyter_output><empty_output><jupyter_text>Linear Regression on Boston House Prices<jupyter_code>import sklearn.datasets as data
boston = data.load_boston()
boston.data.shape, boston.target.shape
boston.target.reshape(-1, 1).shape
import numpy as np
boston_all = np.concatenate((boston.data, boston.target.reshape(-1, 1)), axis=1)
boston_all.shape
print(boston.DESCR)
# median prices of the first 5 suburbs
boston.target[0:4]
# all features of the first 5 suburbs
boston.data[0:4]
# RM       average number of rooms per dwelling
X = boston.data[:, 5:6]
y = boston.target
# just rooms
X[0:4]
y[0:4]
import matplotlib.pyplot as plt

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlim(0, 10) 
plt.ylim(0, 60)
plt.xlabel('average number of rooms')
plt.ylabel('price in $1000\'s')
import numpy as np

# guessing a line that could fit through that

a = 15.; b = -70.
lx = np.arange(4,10)
lguess = a*lx + b

plt.plot(lx, lguess, c='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlim(0, 10) 
plt.ylim(0, 60)
plt.xlabel('average number of rooms')
plt.ylabel('price in $1000\'s')<jupyter_output><empty_output><jupyter_text>Stochastic Gradient DescentUses a loss in relation to the parameter (mean squared error mostly) to optimize the parameters using gradient descent. * https://scikit-learn.org/stable/modules/sgd.html* https://scikit-learn.org/stable/modules/sgd.htmlregression* https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDRegressor.htmlsklearn.linear_model.SGDRegressor<jupyter_code>from sklearn.linear_model import SGDRegressor 

est = SGDRegressor()
est
est.fit(X, y)
est.n_iter_
est.coef_
est.intercept_
# you can use those as a and b 

a = est.coef_; b = est.intercept_
lx = np.arange(4,10)
lguess = a*lx + b

plt.plot(lx, lguess, c='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlim(0, 10) 
plt.ylim(0, 60)
plt.xlabel('average number of rooms')
plt.ylabel('price in $1000\'s')
# or let the est make a prediction for you

y_pred = est.predict(X)
y_pred[0:4]

plt.scatter(X, y_pred, marker='.', alpha=0.5, color='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlim(0, 10) 
plt.ylim(0, 60)
plt.xlabel('average number of rooms')
plt.ylabel('price in $1000\'s')
# R^2 score, the closer to 1 the better
# loosely speaking: how much better is this than predicting the constant mean
# 0 would mean just as good
# neg. would mean even worse
# https://en.wikipedia.org/wiki/Coefficient_of_determination

est.score(X, y)
# we can do better if we disable early stopping
est = SGDRegressor(tol=None)
est.fit(X, y)
est.n_iter_
est.score(X, y)
y_pred = est.predict(X)
plt.scatter(X, y_pred, marker='.', alpha=0.5, color='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlim(0, 10) 
plt.ylim(0, 60)
plt.xlabel('average number of rooms')
plt.ylabel('price in $1000\'s')
cm = np.corrcoef(boston_all.T)
cm.shape

import seaborn as sns

plt.figure(figsize=(20, 20))
hm = sns.heatmap(cm,
                cbar=True,
                annot=True,
                square=True,
                fmt='.2f',
                yticklabels=np.append(boston.feature_names, 'MEDV'),
                xticklabels=np.append(boston.feature_names, 'MEDV'))<jupyter_output><empty_output><jupyter_text>Exercise: Choose a different feafure to predict the target* Execute the notebook up to this point* what would be a good way to choose a feature that is most promising for a prediction?* if you want an advice try `crime rate` STOP HERE  Crime rate<jupyter_code>#         - CRIM     per capita crime rate by town
#         - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
#         - INDUS    proportion of non-retail business acres per town
#         - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
#         - NOX      nitric oxides concentration (parts per 10 million)
#         - RM       average number of rooms per dwelling
#         - AGE      proportion of owner-occupied units built prior to 1940
#         - DIS      weighted distances to five Boston employment centres
#         - RAD      index of accessibility to radial highways
#         - TAX      full-value property-tax rate per $10,000
#         - PTRATIO  pupil-teacher ratio by town
#         - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
#         - LSTAT    % lower status of the population
X = boston.data[:, [0]]
y = boston.target
X[0:4]
est = SGDRegressor(tol=None)
est.fit(X, y)
est.score(X, y)
y_pred = est.predict(X)
plt.scatter(X, y_pred, marker='.', alpha=0.5, color='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlim(0, 10) 
plt.ylim(0, 60)
plt.xlabel('crime rate')
plt.ylabel('price in $1000\'s')<jupyter_output><empty_output><jupyter_text>% lower status of the population<jupyter_code>X = boston.data[:, [12]]
y = boston.target
est = SGDRegressor(tol=None)
est.fit(X, y)
est.score(X, y)
y_pred = est.predict(X)
plt.scatter(X, y_pred, marker='.', alpha=0.5, color='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
# plt.xlim(0, 10) 
# plt.ylim(0, 60)
plt.xlabel('% lower status of the population')
plt.ylabel('price in $1000\'s')<jupyter_output><empty_output><jupyter_text>pupil-teacher ratio by town<jupyter_code>X = boston.data[:, [10]]
y = boston.target
est = SGDRegressor(tol=None)
est.fit(X, y)
est.score(X, y)
y_pred = est.predict(X)
plt.scatter(X, y_pred, marker='.', alpha=0.5, color='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
# plt.xlim(0, 10) 
# plt.ylim(0, 60)
plt.xlabel('pupil-teacher ratio by town')
plt.ylabel('price in $1000\'s')<jupyter_output><empty_output><jupyter_text>Apply Regression to our Insurance Example<jupyter_code>import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/DJCordhose/ml-workshop/master/notebooks/classic/insurance-customers-1500.csv', sep=';')
df.describe()
X = df[['age']]
y = df[['speed']]
import matplotlib.pyplot as plt

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')

from sklearn.linear_model import SGDRegressor 

est = SGDRegressor(tol=None)
est.fit(X, y)
est.score(X, y)
y_pred = est.predict(X)
plt.scatter(X, y_pred, marker='.', alpha=0.5, color='red')

plt.scatter(X, y, marker='.', alpha=0.5, color='blue')
plt.xlabel('age')
plt.ylabel('max speed')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14458732057416268, 'fraction_numerical': 0.0291566985645933, 'mean_word_length': 4.238057948316366, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '599515', 'license': 'MIT', 'n_tokens_mistral': 2679, 'n_tokens_neox': 2478, 'n_words': 639, 'path': 'notebooks/classic/regression.ipynb', 'repo_name': 'kmunve/ml-workshop', 'size': '6688'}