<jupyter_start><jupyter_text>DAY 23 -- Mar 19, 2017<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

%matplotlib inline
from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>A new **KAGGLE** data set to play with:[House Prices: Advanced Regression Techniques:](https://www.kaggle.com/c/house-prices-advanced-regression-techniques) *Predict sales prices and practice feature engineering, RFs, and gradient boosting*The goal of this kaggle competition is to predict the final price of each home from 79 explanatory variables describing (almost) every aspect of residential homes in Ames, Iowa. Load data<jupyter_code>input_dir = "./data/"

# Load my data
train_file = os.path.join(input_dir, "train.csv")
test_file = os.path.join(input_dir, "test.csv")

train = pd.read_csv(train_file)
test = pd.read_csv(test_file)
train.head()
train.columns<jupyter_output><empty_output><jupyter_text>FeaturesAt a first pass, we will only look at quantitative variables.<jupyter_code>features = train.columns[train.dtypes==int]
features = [f for f in features if f not in ["Id", "SalePrice"]]

data = train[features + ["SalePrice"]]
X = train[features]
y = train["SalePrice"]

X.shape, y.shape<jupyter_output><empty_output><jupyter_text>Linear regression using all quantitative featuresWhen I think of predicting a continuous variable, I think of regression (eg. linear regression)<jupyter_code>from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score

clf = LinearRegression()
scores = cross_val_score(clf, X, y, cv=3)

# Print scores
print("The scores:", scores)
print("Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() * 2))<jupyter_output>The scores: [ 0.84792899  0.78651395  0.69920021]
Accuracy: 0.78 (+/- 0.12)<jupyter_text>How well do the quantitative features correlate with sales price?<jupyter_code>corrmat = data.corr()

f, ax = plt.subplots(figsize=(12, 9))
sns.heatmap(corrmat, vmax=.8, square=True);<jupyter_output><empty_output><jupyter_text>- When we look at it, "OverallQual" is highly correlated with SalePrice<jupyter_code>f, (ax1, ax2) = plt.subplots(figsize=(12, 5), ncols=2, nrows=1)
sns.regplot(x="OverallQual", y="SalePrice", data=data, ax=ax1)
sns.boxplot(x="OverallQual", y="SalePrice", data=data, ax=ax2)
# Predicting using only "OverallQual"
clf = LinearRegression()
scores = cross_val_score(clf, X[["OverallQual"]], y, cv=3)

# Print scores
print("The scores:", scores)
print("Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() * 2))<jupyter_output>The scores: [ 0.65298503  0.61625302  0.60493919]
Accuracy: 0.62 (+/- 0.04)<jupyter_text>Considering only highly correlated features (> 0.25 or < -0.25)<jupyter_code>df = pd.DataFrame(corrmat.SalePrice.sort_values())
df.plot.bar(legend=False)
plt.ylabel("Correlation with SalePrice")
plt.hlines(y=0.25, xmin=-10, xmax=100, color="red")
highly_corr_features = corrmat.SalePrice.sort_values()
highly_corr_features = highly_corr_features[abs(highly_corr_features) > .25]
highly_corr_features = [f for f in highly_corr_features.index if f != 'SalePrice']
highly_corr_features
clf = LinearRegression()
scores = cross_val_score(clf, X[highly_corr_features], y, cv=3)

# Print scores
print("The scores:", scores)
print("Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() * 2))<jupyter_output>The scores: [ 0.84424068  0.77671438  0.69100392]
Accuracy: 0.77 (+/- 0.13)
{'fraction_non_alphanumeric': 0.13708058503011183, 'fraction_numerical': 0.044737596788069975, 'mean_word_length': 6.484978540772532, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '134843', 'license': 'MIT', 'n_tokens_mistral': 1394, 'n_tokens_neox': 1246, 'n_words': 302, 'path': 'house_prices/day23-explore-seaborn-linearregression.ipynb', 'repo_name': 'csiu/kaggle', 'size': '3487'}