<jupyter_start><jupyter_text>Multiple Linear Regression - Demo---A multiple linear regression is modelling the relationship between feature vector (X) and its label (y) in a linear approach.(y = a * X1 + b * X2 + ... + C; a, b, ... = slopes for feature Xn, C = constant) Pros*   Works on any size of dataset*   Gives information about the relevance of feature Cons*   Difficult to visualize above three dimensionsand assumes the following:*   Linear relationship*   Multivariate normality*   Little to no multicollinearity*   No auto-correction*   Homoscedasticity---  1. Import dependencies and data<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

!wget https://raw.githubusercontent.com/NicoEssi/Machine_Learning_scikit-learn/master/50_Startups.csv<jupyter_output>--2019-08-23 19:09:03--  https://raw.githubusercontent.com/NicoEssi/Machine_Learning_scikit-learn/master/50_Startups.csv
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 172.16.58.3, 172.16.17.32, 172.16.58.3, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|172.16.58.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2436 (2.4K) [text/plain]
Saving to: ‘50_Startups.csv’


50_Startups.csv       0%[                    ]       0  --.-KB/s               
50_Startups.csv     100%[===================>]   2.38K  --.-KB/s    in 0s      

2019-08-23 19:09:03 (40.2 MB/s) - ‘50_Startups.csv’ saved [2436/2436]<jupyter_text>2. Read CSV and inspect<jupyter_code>data = pd.read_csv("./50_Startups.csv")

X = data.iloc[:, :-1]
y = data.iloc[:, 4]

X = X.values # Conversion due to 'Series' object of pandas.core.series module has no attribute 'reshape'

data.head()<jupyter_output><empty_output><jupyter_text>3. Encode categorical data<jupyter_code>from sklearn.preprocessing import LabelEncoder, OneHotEncoder

labelencoder_X = LabelEncoder()
onehotencoder = OneHotEncoder(categorical_features = [3]) #categorical-features keyboard is deprecated; use ColumnTransformer instead in v0.22+
X[:, 3] = labelencoder_X.fit_transform(X[:, 3])
X = onehotencoder.fit_transform(X).toarray()

# Addressing the dummy variable trap
X = X[:, 1:]
# though not necessary, as the library handles this automatically<jupyter_output><empty_output><jupyter_text>4. Feature scaling<jupyter_code>from sklearn.preprocessing import StandardScaler

sc_X = StandardScaler()
X[:, 2:5] = sc_X.fit_transform(X[:, 2:5])

# though not necessary either, as the library does this anyway<jupyter_output><empty_output><jupyter_text>5. Split the dataset into training and test set<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)<jupyter_output><empty_output><jupyter_text>6. Initialize and fit the multiple regression model<jupyter_code>from sklearn.linear_model import LinearRegression

regressor = LinearRegression()
regressor.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>7. Predict with the trained model<jupyter_code>y_predictions = regressor.predict(X_test)<jupyter_output><empty_output><jupyter_text>8. Optimize with backward elimination (p-value = 0.050)<jupyter_code>import statsmodels.api as sm

# Add bias as it is not accounted for by statsmodel library
X = np.append(arr = np.ones((50,1)).astype(int), values = X, axis = 1)
X_optimal = X[:, [0, 1, 2, 3, 4, 5]]
regressor_OLS = sm.OLS(endog = y, exog = X_optimal).fit()
regressor_OLS.summary()
# remove x2 as P>|t| = 0.990

X_optimal = X[:, [0, 1, 3, 4, 5]]
regressor_OLS = sm.OLS(endog = y, exog = X_optimal).fit()
regressor_OLS.summary()
# remove x1 as P>|t| = 0.940

X_optimal = X[:, [0, 3, 4, 5]]
regressor_OLS = sm.OLS(endog = y, exog = X_optimal).fit()
regressor_OLS.summary()
# remove x2 as P>|t| = 0.602

X_optimal = X[:, [0, 3, 5]]
regressor_OLS = sm.OLS(endog = y, exog = X_optimal).fit()
regressor_OLS.summary()
# remove x2 as P>|t| = 0.060.

X_optimal = X[:, [0, 3]]
regressor_OLS = sm.OLS(endog = y, exog = X_optimal).fit()
regressor_OLS.summary()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13242453748782862, 'fraction_numerical': 0.0440603700097371, 'mean_word_length': 4.903735632183908, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '421964', 'license': 'MIT', 'n_tokens_mistral': 1621, 'n_tokens_neox': 1465, 'n_words': 372, 'path': 'Multiple_Linear_Regression_Demo.ipynb', 'repo_name': 'NicoEssi/Machine_Learning_Python', 'size': '4118'}