<commit_before>---
layout: single
title:  "House Sale Prices in Ames, Iowa"
date:   2017-10-26 
excerpt: Advanced regression techniques, Python
tags: 
  - Regression
  - Ensemble and Stacking
comments: true  
---

Here, the goal is to investigate [Kaggle's House Price competition](https://www.kaggle.com/c/house-prices-advanced-regression-techniques). I try different regression models inclduing boosting methods (see [here](https://en.wikipedia.org/wiki/Boosting_(machine_learning)) based on sklearn package. 


```python
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline  

directory = '../../Datasets/House_Price/'
train_input = pd.read_csv(directory+'train.csv')
test_input = pd.read_csv(directory+'test.csv')
test_ID = test_input['Id']
```

### Correlation between columns

First, let's generate a heatmap to evaluate the correlation between Sale Price and other paramters. Clearly, some paramters such as LotArea and LotFrontage are very important and they are highly correclated with the final Sale Price.


```python
# generate a headmap for features in a data frame
import seaborn as sns
import numpy as np
corr = train_input.corr()
# Generate a mask for the upper triangle
mask = np.zeros_like(corr, dtype=np.bool)
mask[np.triu_indices_from(mask)] = True
f, ax = plt.subplots(figsize=(15, 12))

# Generate a custom diverging colormap
cmap = sns.diverging_palette(220, 10, as_cmap=True)
sns.heatmap(train_input.corr(), mask=mask, cmap=cmap, vmax=.3, center=0,
            square=True, linewidths=.5, cbar_kws={"shrink": .5})
```




    <matplotlib.axes._subplots.AxesSubplot at 0x9839f98>




![png](../figures/house_price_predictor/output_4_1.png)


### Removing outliers


```python
hsize = 2
wsize = 2
f, ax = plt.subplots(hsize,wsize,figsize=[10, 10])
columns = ['GrLivArea', 'LotFrontage', 'LotArea', 'TotalBsmtSF']
for i in range(0,hsize):
    for j in range(0,wsize):
        train_input.plot.scatter(columns[i*wsize+j],'SalePrice', ax=ax[i][j], title=None)
plt.tight_layout() 
```


![png](../figures/house_price_predictor/output_6_0.png)



```python
train_input.shape
```




    (1460, 81)




```python
train_input = train_input.drop(train_input[(train_input['GrLivArea']>4000) & (train_input['SalePrice']<300000)].index)
train_input = train_input.drop(train_input[(train_input['LotFrontage']>300) & (train_input['SalePrice']<300000)].index)
train_input = train_input.drop(train_input[(train_input['LotArea']>100000) & (train_input['SalePrice']<400000)].index)
train_input.shape
```




    (1453, 81)




```python
hsize = 2
wsize = 2
f, ax = plt.subplots(hsize,wsize,figsize=[10, 10])
columns = ['GrLivArea', 'LotFrontage', 'LotArea', 'TotalBsmtSF']
for i in range(0,hsize):
    for j in range(0,wsize):
        train_input.plot.scatter(columns[i*wsize+j],'SalePrice', ax=ax[i][j], title=None)
plt.tight_layout() 
```


![png](../figures/house_price_predictor/output_9_0.png)


### Rescaling target values
<commit_msg>fixed typo<commit_after>---
layout: single
title:  "House Sale Prices in Ames, Iowa"
date:   2017-10-26 
excerpt: Advanced regression techniques, Python
tags: 
  - Regression
  - Ensemble and Stacking
comments: true  
---

Here, the goal is to investigate [Kaggle's House Price competition](https://www.kaggle.com/c/house-prices-advanced-regression-techniques). I try different regression models inclduing boosting methods (see [here](https://en.wikipedia.org/wiki/Boosting_(machine_learning)) based on sklearn package. 


```python
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline  

directory = '../../Datasets/House_Price/'
train_input = pd.read_csv(directory+'train.csv')
test_input = pd.read_csv(directory+'test.csv')
test_ID = test_input['Id']
```

### Correlation between columns

First, let's generate a heatmap to evaluate the correlation between Sale Price and other paramters. Clearly, some paramters such as LotArea and LotFrontage are very important and they are highly correclated with the final Sale Price.


```python
# generate a headmap for features in a data frame
import seaborn as sns
import numpy as np
corr = train_input.corr()
# Generate a mask for the upper triangle
mask = np.zeros_like(corr, dtype=np.bool)
mask[np.triu_indices_from(mask)] = True
f, ax = plt.subplots(figsize=(15, 12))

# Generate a custom diverging colormap
cmap = sns.diverging_palette(220, 10, as_cmap=True)
sns.heatmap(train_input.corr(), mask=mask, cmap=cmap, vmax=.3, center=0,
            square=True, linewidths=.5, cbar_kws={"shrink": .5})
```




    <matplotlib.axes._subplots.AxesSubplot at 0x9839f98>




![png](../figures/House_Price_Predictor/output_4_1.png)


### Removing outliers


```python
hsize = 2
wsize = 2
f, ax = plt.subplots(hsize,wsize,figsize=[10, 10])
columns = ['GrLivArea', 'LotFrontage', 'LotArea', 'TotalBsmtSF']
for i in range(0,hsize):
    for j in range(0,wsize):
        train_input.plot.scatter(columns[i*wsize+j],'SalePrice', ax=ax[i][j], title=None)
plt.tight_layout() 
```


![png](../figures/House_Price_Predictor/output_6_0.png)



```python
train_input.shape
```




    (1460, 81)




```python
train_input = train_input.drop(train_input[(train_input['GrLivArea']>4000) & (train_input['SalePrice']<300000)].index)
train_input = train_input.drop(train_input[(train_input['LotFrontage']>300) & (train_input['SalePrice']<300000)].index)
train_input = train_input.drop(train_input[(train_input['LotArea']>100000) & (train_input['SalePrice']<400000)].index)
train_input.shape
```




    (1453, 81)




```python
hsize = 2
wsize = 2
f, ax = plt.subplots(hsize,wsize,figsize=[10, 10])
columns = ['GrLivArea', 'LotFrontage', 'LotArea', 'TotalBsmtSF']
for i in range(0,hsize):
    for j in range(0,wsize):
        train_input.plot.scatter(columns[i*wsize+j],'SalePrice', ax=ax[i][j], title=None)
plt.tight_layout() 
```


![png](../figures/House_Price_Predictor/output_9_0.png)


### Rescaling target values

{'fraction_non_alphanumeric': 0.1451476793248945, 'fraction_numerical': 0.03139240506329114, 'mean_word_length': 5.172916666666667, 'pattern_counts': {'":': 2, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6190251', 'n_tokens_mistral': 2345, 'n_tokens_neox': 2080, 'n_words': 481}