<jupyter_start><jupyter_text>Let's build a model!  We'll use a data set that is standard in scikit-learn.<jupyter_code>import pandas as pd
from sklearn.datasets import fetch_california_housing
data = fetch_california_housing()<jupyter_output><empty_output><jupyter_text>This will give us a description of what's in this data set.<jupyter_code>print(data.DESCR)<jupyter_output>.. _california_housing_dataset:

California Housing dataset
--------------------------

**Data Set Characteristics:**

    :Number of Instances: 20640

    :Number of Attributes: 8 numeric, predictive attributes and the target

    :Attribute Information:
        - MedInc        median income in block
        - HouseAge      median house age in block
        - AveRooms      average number of rooms
        - AveBedrms     average number of bedrooms
        - Population    block population
        - AveOccup      average house occupancy
        - Latitude      house block latitude
        - Longitude     house block longitude

    :Missing Attribute Values: None

This dataset was obtained from the StatLib repository.
http://lib.stat.cmu.edu/datasets/

The target variable is the median house value for California districts.

This dataset was derived from the 1990 U.S. census, using one row per census
block group. A block group is the smallest geographical unit for which the U.S.
Census Bur[...]<jupyter_text>So let's get the data into a pandas DataFrame (X) and a Series (y).<jupyter_code>X = pd.DataFrame(data['data'], columns=data['feature_names'])
X.head()
y = data['target']
y<jupyter_output><empty_output><jupyter_text>A tiny bit of exploratory data analysis.  Let's draw a plot of MedInc vs. Average Price.<jupyter_code>import matplotlib.pyplot as plt
plt.scatter(X['MedInc'], y)
plt.xlabel('Median Income')
plt.ylabel('Average Price')
plt.show()
print(y.min(), y.max())<jupyter_output>0.14999 5.00001<jupyter_text>The data set is a little strange, as the Average Price is clearly capped at 5.  What can we do?  We dont' know why, so let's just continue with what we want to do, build a model (or several models) to try to predict the Average Value using other features in the data set.  We want to build a model such as the following one:      ave price = a*(median income) + b What is a and b?  What values of a and b will give me a "good" fit to this data?  The idea is to find these values of a and b, and then we can use this to predict other house prices given the median income of other neighborhoods.<jupyter_code>from sklearn.linear_model import LinearRegression<jupyter_output><empty_output><jupyter_text>Linear regression is usually a good "benchmark" to first consider as it's easy to fit a model, and gives us a "first effort" to compare other models against.<jupyter_code>lr = LinearRegression()

lr.fit(X['MedInc'].values.reshape(-1,1), y)
X['MedInc'].values.reshape(-1,1)<jupyter_output><empty_output><jupyter_text>This cell below has the same outcome as the previous one.  The main thing to keep in mind is that the feature matrix needs to be given to scikit-learn in the proper format, namely a two-dimensional "array".  Rows correspond to observations, columns correspond to features of our data.  In this example there is only one feature.  Scikit-learn can use numpy arrays, lists of lists, or pandas DataFrames.  The previous cell got the data in the form of a numpy array.  The cell below gives the data in the form of a DataFrame.  (Remember that uses double brackets will return a DataFrame!)<jupyter_code>X[['MedInc']].head()
lr.fit(X[['MedInc']], y)<jupyter_output><empty_output><jupyter_text>We can get the coefficient and intercept of this model.  In terms of "a" and "b" used before, "a" is the coefficient, and "b" is the intercept.  The coefficients are a numpy array, but there is only one element in this numpy array for this model as there is only a single feature.<jupyter_code>print(f'Coefficient: {lr.coef_[0]}')
print(f'Intercept: {lr.intercept_}')<jupyter_output>Coefficient: 0.41793849201896277
Intercept: 0.45085576703267893<jupyter_text>ave price = 0.418*(median income) + 0.451 Every predictor in scikit-learn has a `predict` method to get predictions.  There is also a `score` function.  For a regressor object (such as `LinearRegression`) this gives us the so-called "R-squared" value.<jupyter_code>lr.predict(X[['MedInc']].head(10))
lr.score(X[['MedInc']], y)<jupyter_output><empty_output><jupyter_text>We can manually figure the score ourselves using the predict function together with other `numpy` methods.<jupyter_code>1 - ((lr.predict(X[['MedInc']]) - y)**2).sum()/((y.mean() - y)**2).sum()<jupyter_output><empty_output><jupyter_text>We can also create a new linear regression using all of the features, just as easy as using a single feature.<jupyter_code>lr_full = LinearRegression()
lr_full.fit(X, y)
lr_full.score(X, y)
lr_full.coef_
lr_full.intercept_
X.columns<jupyter_output><empty_output><jupyter_text>Our new model looks something like this.  There are eight terms, one for each feature and the constant (intercept).ave price = 0.437\*MedInc + 0.00944\*HouseAge - 0.108\*AveRooms + ......... - 36.94<jupyter_code>print('Median House Price =')
print(f'   {lr_full.intercept_}')
for c, v in zip(lr_full.coef_, X.columns):
    print(f'   {c} x {v} +')
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X)
scaler.mean_
X.columns
scaler.scale_
scaler.transform(X)
X_transformed = scaler.transform(X)
X_transformed.mean(axis=0)
X_transformed.mean(axis=1)
X_transformed.mean()
X_transformed.std(axis=0)
lr = LinearRegression()
lr.fit(X_transformed, y)
lr.coef_
import numpy as np 
np.abs(lr.coef_)
print('Median House Price = ')
print(f'   {lr.intercept_}')
for c, v in zip(lr.coef_, X.columns):
    print(f'   {c} x (transformed {v}) +')<jupyter_output>Median House Price = 
   2.0685581690891386
   0.82961930428045 x (transformed MedInc) +
   0.1187516512121425 x (transformed HouseAge) +
   -0.26552687950661974 x (transformed AveRooms) +
   0.30569622980430833 x (transformed AveBedrms) +
   -0.0045030013126133445 x (transformed Population) +
   -0.039326266978148866 x (transformed AveOccup) +
   -0.8998856544145062 x (transformed Latitude) +
   -0.8705410023357302 x (transformed Longitude) +<jupyter_text>Typically we start to put our transformers (and final estimator, perhaps) into a scikit-learn Pipeline.  This makes it easier to ensure that we do all of the transformations, and do them in the correct order.  It helps to make the whole process easier.  It also makes it easier to share our resulting model with someone else.<jupyter_code>from sklearn.pipeline import Pipeline<jupyter_output><empty_output><jupyter_text>A Pipeline consists of a number of steps that are given as a list of tuples.  The first element in the tuple is a string that can be used to identify the particular step.  The second element in the tuple is a transformer or estimator.  There can be only one estimator, and, if there is one, it has to be at the end of the Pipeline.<jupyter_code>pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('lr', LinearRegression())
])<jupyter_output><empty_output><jupyter_text>A Pipeline has a `fit` method.  That calls `fit_trasform` on the transformers.  The output of one transformer becomes the input to the next transformer.  If there is an estimator, the output from the final transformer becomes the input (the feature matrix) to the `fit` method of the estimator.<jupyter_code>pipe.fit(X, y)<jupyter_output><empty_output><jupyter_text>The `predict` method of the Pipeline will call the `transform` method of each of the transformers.  Then the output of the transformers will be fed into the `predict` method of the final estimator (assuming there is one, otherwise it makes no sense to call `predict` on the Pipeline).<jupyter_code>pipe.predict(X.head(5))<jupyter_output><empty_output><jupyter_text>The `score` method of the Pipeline will be whatever the `score` method is for the estimator in the Pipeline.  In this case, since we are using `LinearRegression`, the score will be the $R^2$ value.<jupyter_code>pipe.score(X, y)<jupyter_output><empty_output><jupyter_text>We can also access the individual steps of the Pipeline.  The `named_steps` attribute is a dictionary, where the keys are the strings used to define the Pipeline.  So using this, we can also access the attributes of the underlying pieces in the Pipeline, such as the intercept and coefficients of the `LinearRegression` class here.<jupyter_code>pipe.named_steps['lr'].intercept_
pipe.named_steps['lr'].coef_
pipe.named_steps['scaler'].transform(X.head(10))
pipe.named_steps['scaler'].mean_<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09376793297371744, 'fraction_numerical': 0.02984046826580971, 'mean_word_length': 4.809333333333333, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '166545', 'license': 'Unlicense', 'n_tokens_mistral': 2929, 'n_tokens_neox': 2666, 'n_words': 1020, 'path': 'MachineLearning/TrackingStudyAtWQ/FromTeacher/Office-Hours-4.ipynb', 'repo_name': 'jash-kothari-forks/SharingKnowledge', 'size': '8713'}