<jupyter_start><jupyter_text>Let's see the simple code for Linear Regression.We will be creating a model to predict weight of a person based on independent variable height using simple linear regression.weight-height dataset is downloaded from kagglehttps://www.kaggle.com/sonalisingh1411/linear-regression-using-weight-height/data<jupyter_code>from google.colab import drive
drive.mount('/content/drive')
%cd /content/drive/My\ Drive
%cd 'Colab Notebooks'<jupyter_output>/content/drive/My Drive/Colab Notebooks<jupyter_text>1. Simple Linear Regression with one independent variable We will read the data file and do some data exploration.<jupyter_code>import numpy as np
import pandas as pd 
import matplotlib.pyplot as plt

df = pd.read_csv('weight-height.csv')
df.head()
df.shape
df.columns
df['Gender'].unique()
df.corr()<jupyter_output><empty_output><jupyter_text>We can see that there is high co-relation between height and weight columns.We will use Linear Regression model from sklearn library<jupyter_code>x = df['Height']
y = df['Weight']<jupyter_output><empty_output><jupyter_text>We will split the data into train and test datasets using sklearn preprocessing library<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(x,y,test_size=0.2, random_state=42)
X_train.shape
X_train = X_train.to_numpy()
X_train = X_train.reshape(-1,1)<jupyter_output><empty_output><jupyter_text>reshape() is called to make X_train 2-dimesional that is row and column format<jupyter_code>X_train.shape
X_test = X_test.to_numpy()
X_test = X_test.reshape(-1,1)
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train,y_train)<jupyter_output><empty_output><jupyter_text>model is created as instnace of  LinearRegression. With .fit() method, optimal values of coefficients (b0,b1) are calculated using existing input X_train and y_train.<jupyter_code>model.score(X_train,y_train)<jupyter_output><empty_output><jupyter_text>The arguments to .score() are also X_train and y_train and it returns the R2 (coefficient of determination).<jupyter_code>Intercept,coef = model.intercept_,model.coef_
print("Intercept is :",Intercept, sep='\n')
print("Coefficient/slope is :",coef , sep='\n')<jupyter_output>Intercept is :
-349.7878205824451
Coefficient/slope is :
[7.70218561]<jupyter_text>Model attributes model.intercept_, model.coef_ give the value of (b01,b1) Now, we will use trained model to predict on test data<jupyter_code>y_pred = model.predict(X_test)
y_pred<jupyter_output><empty_output><jupyter_text>We can use also slope-intercept of line y = y-intercept + slope * x to predict the values on test data. We will use model.intercept_ and model.coef_ value for predictiion<jupyter_code>y_pred1 = Intercept + coef * X_test
y_pred1<jupyter_output><empty_output><jupyter_text>We can see output of both y_pred and y_pred1 is same. We will plot the graph of predicted and actual values of weights using seaborn and matplotlib library<jupyter_code>import seaborn as sns
ax = sns.regplot(x=y_pred, y=y_test,
                 x_estimator=np.mean)<jupyter_output><empty_output><jupyter_text>To clearly see the the plot, let's draw 20 samples from training dataset with actual weight values and plot it with predicted weight values for training dataset.The red dots represent the actual weight values(20 samples drawn) and the green line represents the predcted weight values by the model. The vertical distance between red dot and the green line is the error which we have to minimize to best fit the model.<jupyter_code>plt.scatter(X_train[0:20], y_train[0:20], color = "red")
plt.plot(X_train, model.predict(X_train), color = "green")
plt.title("Weight vs Height")
plt.xlabel("Height")
plt.ylabel("Weight")
plt.show()<jupyter_output><empty_output><jupyter_text>2. Multiple Linear Regressiom A regression with 2 or more independet variables is multiple linear regression. We will use same dataset to implemet multiple linear regression.The 2 independent variables will be gender and height which be used to predict the weight.<jupyter_code>x = df.drop(columns = 'Weight')
y = df['Weight']
x.columns<jupyter_output><empty_output><jupyter_text>Gender column is categorical. We can not use it directly as model can work only with numbers. We have to convert it to one-hot-encoding using pandas get_dummies() method. A new column will be create dropping earlier column . The new column contain values 1 and 0 for male and female respectively.<jupyter_code>x = pd.get_dummies(x, columns = ['Gender'], drop_first = True)
x
print(x.shape)
print(y.shape)<jupyter_output>(10000, 2)
(10000,)<jupyter_text>Rest of the steps will be same as simple linear regression.<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(x,y,test_size=0.2, random_state=42)
X_train = X_train.to_numpy()
X_train = X_train.reshape(-1,2)

X_test = X_test.to_numpy()
X_test = X_test.reshape(-1,2)
X_train.shape
mulLR = LinearRegression()
mulLR.fit(X_train,y_train)
mulLR.score(X_train,y_train)
Intercept,coef = mulLR.intercept_,mulLR.coef_
print("Intercept is :",Intercept, sep='\n')
print("Coefficient/slope is :",coef , sep='\n')<jupyter_output>Intercept is :
-244.69356793639193
Coefficient/slope is :
[ 5.97314123 19.34720343]<jupyter_text>Coefficient array will have 2 values for gender and height respectively.<jupyter_code>y_pred = mulLR.predict(X_test)
y_pred<jupyter_output><empty_output><jupyter_text>Alternate method : Predicting weight using coefficient and intercept values in equation<jupyter_code>y_pred1 = Intercept + np.sum(coef * X_test, axis = 1)
y_pred1<jupyter_output><empty_output><jupyter_text>y_pred and y_pred1 both have same predicted values<jupyter_code>import seaborn as sns
ax = sns.regplot(x=y_pred, y=y_test,
                 x_estimator=np.mean)<jupyter_output><empty_output><jupyter_text>Above plot shows graph representing predicted and actual weight values on test dataset. 3. Polynomial Regression We will use polynomial regression to find the weight using same dataset. Note that polynomial regression is the special case of linear regression. Import class PolynomialFeatures from sklearn.preprocessing<jupyter_code>from sklearn.preprocessing import PolynomialFeatures
x = df['Height']
y = df['Weight']
transformer = PolynomialFeatures(degree = 2, include_bias = False)<jupyter_output><empty_output><jupyter_text>We have to include terms like x2(x squared) as additional features when using polynomial regression.We have to transform the inputfor that transformer is defined with degree (defines the degree of polynomial regression function) and include_bias decides whether to include bias or not.<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(x,y,test_size=0.2, random_state=42)
X_train = X_train.to_numpy()
X_train = X_train.reshape(-1,1)

X_test = X_test.to_numpy()
X_test = X_test.reshape(-1,1)
transformer.fit(X_train) 
X_trans = transformer.transform(X_train)<jupyter_output><empty_output><jupyter_text>Above two lines of code can be fit into one line as below, both will give same output<jupyter_code>x_trans = PolynomialFeatures(degree=2, include_bias=False).fit_transform(X_train)

X_transtest = PolynomialFeatures(degree=2, include_bias=False).fit_transform(X_test)<jupyter_output><empty_output><jupyter_text>Each value in the first column is squared and stored in second column as feature<jupyter_code>print(x_trans)<jupyter_output>[[  61.39164365 3768.93390949]
 [  74.6976372  5579.7370037 ]
 [  68.50781491 4693.32070353]
 ...
 [  64.3254058  4137.75783102]
 [  69.07449203 4771.28544943]
 [  67.58883983 4568.25126988]]<jupyter_text>Create and fit the model<jupyter_code>poly_LR = LinearRegression().fit(x_trans,y_train)
poly_LR.score(x_trans,y_train)
y_pred = poly_LR.predict(X_transtest)
y_pred
Intercept,coef = mulLR.intercept_,mulLR.coef_
print("Intercept is :",Intercept, sep='\n')
print("Coefficient/slope is :",coef , sep='\n')<jupyter_output>Intercept is :
-244.69356793639193
Coefficient/slope is :
[ 5.97314123 19.34720343]<jupyter_text>The score of ploynomial regression can slighly be better than linear regression due to added complexity but the high R2 scoe does not always mean good model. Sometimes ploynomial regression could lead to overfitting due to its complexity in defining the equation for regression.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0893926935275461, 'fraction_numerical': 0.035475155644308705, 'mean_word_length': 6.642728904847397, 'pattern_counts': {'":': 0, '<': 87, '<?xml version=': 0, '>': 87, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '668684', 'license': 'MIT', 'n_tokens_mistral': 3062, 'n_tokens_neox': 2818, 'n_words': 800, 'path': 'Linear_Regression.ipynb', 'repo_name': 'tejashrigadre/Linear_Regression', 'size': '8513'}