<jupyter_start><jupyter_text>Predicting house prices with regression> Regression problems normally answers questions like **How much...?**, **How mayny...?**. In this notebook we will use different **Regression** Algorithms to predict the house prices from the well known dataset in **sklearn** `load_boston`<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
import pandas as pd
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Loading data.<jupyter_code>boston = datasets.load_boston()
pd.DataFrame(boston.data).head()
print(boston)
boston.feature_names
boston.target.shape, boston.data.shape<jupyter_output><empty_output><jupyter_text>Spliting our data> We are goin to split our data into train, and test<jupyter_code>from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
X_train, X_test, y_train, y_test = train_test_split(boston.data, boston.target, test_size = .3, random_state =99)
X_train.shape, y_train.shape, X_test.shape<jupyter_output><empty_output><jupyter_text>Creating different `Regression` models.<jupyter_code>from sklearn.linear_model import SGDRegressor
from sklearn.svm import SVR
from sklearn.ensemble import ExtraTreesRegressor, RandomForestRegressor
from sklearn.neighbors import KNeighborsRegressor
from sklearn.pipeline import Pipeline
from sklearn.tree import DecisionTreeRegressor
from sklearn.linear_model import LinearRegression, LogisticRegression<jupyter_output><empty_output><jupyter_text>Scaling our X and y data for trsin and test> We are going to scale our data using the `StandardScaler` for both train and test datasets.<jupyter_code>scaleX = StandardScaler()
scaley = StandardScaler()
X_train_scaled = scaleX.fit_transform(X_train)
X_test_scaled = scaleX.transform(X_test)
X_train_scaled[0]
y_test[:2]<jupyter_output><empty_output><jupyter_text>1. SGDRegressor> SGD stands for Stochastic Gradient Descent: the gradient of the loss is estimated each sample at a time and the model is updated along the way with a decreasing strength schedule (aka learning rate).<jupyter_code>sgd_reg = SGDRegressor(loss='squared_loss', random_state=42, penalty=None)
sgd_reg
sgd_reg.fit(X_train_scaled, y_train)
sgd_reg.predict(X_test_scaled[:5]), y_test[:5]<jupyter_output><empty_output><jupyter_text>2. SVR> The implementation is based on libsvm. The fit time complexity is more than quadratic with the number of samples which makes it hard to scale to datasets with more than a couple of 10000 samples.<jupyter_code>svr_reg = SVR(C=1, epsilon=.5, kernel="linear",degree=3 )
svr_reg
svr_reg.fit(X_train_scaled, y_train)
svr_reg.predict(X_test_scaled[:5]), y_test[:5]<jupyter_output><empty_output><jupyter_text>3. DecisionTreeRegressor```class sklearn.tree.DecisionTreeRegressor(*, criterion='mse', splitter='best', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, ccp_alpha=0.0)```<jupyter_code>dst_reg = DecisionTreeRegressor()
dst_reg.fit(X_train_scaled, y_train)
dst_reg.predict(X_train_scaled[:5]), y_train[:5]<jupyter_output><empty_output><jupyter_text>4. RandomForestRegressor> A random forest is a meta estimator that fits a number of classifying decision trees on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting. The sub-sample size is controlled with the max_samples parameter if bootstrap=True (default), otherwise the whole dataset is used to build each tree.<jupyter_code>randf_reg = RandomForestRegressor(n_estimators = 5, max_depth=5, random_state=45)

randf_reg.fit(X_train_scaled, y_train)
randf_reg.predict(X_train_scaled[:5]), y_train[:5]<jupyter_output><empty_output><jupyter_text>3. ExtraTreesRegressor> This class implements a meta estimator that fits a number of randomized decision trees (a.k.a. extra-trees) on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting.<jupyter_code>ext_reg = ExtraTreesRegressor(n_estimators=100, random_state=0)
ext_reg.fit(X_train_scaled, y_train)
ext_reg.predict(X_test_scaled[:5]), y_test[:5]<jupyter_output><empty_output><jupyter_text>4. KNeighborsRegressor > The target is predicted by local interpolation of the targets associated of the nearest neighbors in the training set.<jupyter_code>neigh_reg = KNeighborsRegressor(n_neighbors=2)
neigh_reg.fit(X_train_scaled, y_train)
neigh_reg.predict(X_test_scaled[:5]), y_test[:5]<jupyter_output><empty_output><jupyter_text>7. LinearRegression> LinearRegression fits a linear model with coefficients w = (w1, …, wp) to minimize the residual sum of squares between the observed targets in the dataset, and the targets predicted by the linear approximation.<jupyter_code>l_reg = LinearRegression()
l_reg.fit(X_train_scaled, y_train)
l_reg.fit(X_train_scaled, y_train)
l_reg.predict(X_test_scaled[:5]), y_test[:5]<jupyter_output><empty_output><jupyter_text>Scoring Different models<jupyter_code>svr_reg ## SVR
sgd_reg ## SGDRegressor
dst_reg ## DecisionTreeRegressor
l_reg ## LinearRegression
neigh_reg ## KNeighborsRegressor
randf_reg ## RandomForestRegressor
ext_reg   ## ExtraTreesRegressor<jupyter_output><empty_output><jupyter_text>SVR<jupyter_code>svr_reg.score(X_train_scaled, y_train), svr_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output><jupyter_text>> `SVR` is `73%` accurate on the train and `67%` accurate on the test  SGDRegressor<jupyter_code>sgd_reg.score(X_train_scaled, y_train), sgd_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output><jupyter_text>> `SGDRegressor` is `76%` accurate on the train and `67%` accurate on the test  DecisionTreeRegressor<jupyter_code>dst_reg.score(X_train_scaled, y_train), dst_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output><jupyter_text>`DecisionTreeRegressor` is `100%` accurate on the train and `75%` accurate on the test  LinearRegression<jupyter_code>l_reg.score(X_train_scaled, y_train), l_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output><jupyter_text>`LinearRegression` is `76%` accurate on the train and `67%` accurate on the test  KNeighborsRegressor<jupyter_code>neigh_reg.score(X_train_scaled, y_train), neigh_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output><jupyter_text>`KNeighborsRegressor` is `94%` accurate on the train and `71%` accurate on the test  RandomForestRegressor<jupyter_code>randf_reg.score(X_train_scaled, y_train), randf_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output><jupyter_text>`RandomForestRegressor` is `93%` accurate on the train and `77%` accurate on the test  ExtraTreesRegressor<jupyter_code>ext_reg.score(X_train_scaled, y_train), ext_reg.score(X_test_scaled, y_test)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09220477021524141, 'fraction_numerical': 0.011489237929028505, 'mean_word_length': 9.068814055636896, 'pattern_counts': {'":': 0, '<': 81, '<?xml version=': 0, '>': 92, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '159124', 'license': 'MIT', 'n_tokens_mistral': 2467, 'n_tokens_neox': 2355, 'n_words': 553, 'path': 'Predicting House Price Regression (__all__).ipynb', 'repo_name': 'CrispenGari/Machine-Leaning-Scikit-learn', 'size': '6876'}