<jupyter_start><jupyter_text>Regression Week 3: Assessing Fit (polynomial regression) In this notebook you will compare different regression models in order to assess which model fits best. We will be using polynomial regression as a means to examine this topic. In particular you will:* Write a function to return an datafrme where each column is to a polynomial value up to the total degree  e.g. degree = 3, then column 1 is itself. column 2 is the  squared and column 3 is the  cubed* Use matplotlib to visualize polynomial regressions* Use matplotlib to visualize the same polynomial degree on different subsets of the data* Use a validation set to select a polynomial degree* Assess the final fit using test data<jupyter_code>import pandas as pd
import numpy as np
from sklearn import linear_model
import matplotlib.pyplot as plt
%matplotlib inline
tmp = pd.Series([1., 2., 3.])
tmp_cubed = tmp.apply(lambda x: x**3)
print (tmp)
print (tmp_cubed)<jupyter_output>0    1.0
1    2.0
2    3.0
dtype: float64
0     1.0
1     8.0
2    27.0
dtype: float64<jupyter_text>Create a polynomial function<jupyter_code>def polynomial(feature, degree):
    poly_dataframe = pd.DataFrame()
    poly_dataframe['power_1'] = feature

    if degree > 1:
        for power in range(2, degree+1): 
            name = 'power_' + str(power)
            tmp = feature.apply(lambda x: x**power)
            poly_dataframe[name] = tmp

    return poly_dataframe
# test the function:
print (polynomial(tmp, 3))<jupyter_output>power_1  power_2  power_3
0      1.0      1.0      1.0
1      2.0      4.0      8.0
2      3.0      9.0     27.0<jupyter_text>Visualize polynomial regression<jupyter_code>sales = pd.read_csv('kc_house_data.csv')
sales.sort_values(['sqft_living', 'price'])
poly1_data = polynomial(sales['sqft_living'], 1)
poly1_data['price'] = sales['price'] # add price to the data since it's the target
poly1_data<jupyter_output><empty_output><jupyter_text>First degree polynomial regression<jupyter_code>lm = linear_model.LinearRegression()
model1 = lm.fit(poly1_data['power_1'].values.reshape(-1,1), poly1_data['price'])
print(model1.coef_, model1.intercept_)
plt.plot(poly1_data['power_1'],poly1_data['price'],'.',
        poly1_data['power_1'], model1.predict(poly1_data['power_1'].values.reshape(-1,1)),'-')
#plt.plot(poly1_data['power_1'],poly1_data['price'],'.')
#plt.plot(poly1_data['power_1'], model1.predict(poly1_data['power_1'].values.reshape(-1,1)),'-')<jupyter_output><empty_output><jupyter_text>Second degree polynomial regression<jupyter_code>poly2_data = polynomial(sales['sqft_living'], 2)
features = poly2_data.columns
poly2_data['price'] = sales['price']

model = linear_model.LinearRegression()
model.fit(poly2_data[features], poly2_data['price'])

plt.plot(poly2_data['power_1'],poly2_data['price'],'.',
        poly2_data['power_1'], model.predict(poly2_data[features]),'-')
plt.show()
lm = linear_model.LinearRegression()
model2 = lm.fit(poly2_data[features], poly2_data['price'])
print(model2.coef_, model2.intercept_)
plt.plot(poly2_data['power_1'],poly2_data['price'],'.',
        poly2_data['power_1'], model2.predict(poly2_data[features]),'-')
plt.show()
#plt.plot(poly2_data['power_1'],poly2_data['price'],'.')
#plt.plot(poly2_data['power_1'], model2.predict(poly2_data[features]), ',', color = 'red', alpha = 1)<jupyter_output><empty_output><jupyter_text>15th degree polynomial regression<jupyter_code>poly15_data = polynomial(sales['sqft_living'], 15)
features = poly15_data.columns
poly15_data['price'] = sales['price']

poly15_data
lm = linear_model.LinearRegression()
model15 = lm.fit(poly15_data[features], poly15_data['price'])
print(model15.coef_, model15.intercept_)
plt.plot(poly15_data['power_1'],poly15_data['price'],'.',
        poly15_data['power_1'], model15.predict(poly15_data[features]),'-')
#plt.plot(poly15_data['power_1'],poly15_data['price'],'.')
#plt.plot(poly15_data['power_1'], model15.predict(poly15_data[features]), ',', color = 'red')<jupyter_output><empty_output><jupyter_text>Changing the data and re-learning<jupyter_code>sales_set_1 = pd.read_csv('wk3_kc_house_set_1_data.csv')
sales_set_2 = pd.read_csv('wk3_kc_house_set_2_data.csv')
sales_set_3 = pd.read_csv('wk3_kc_house_set_3_data.csv')
sales_set_4 = pd.read_csv('wk3_kc_house_set_4_data.csv')
poly15_data = polynomial(sales_set_1['sqft_living'], 15)
features = poly15_data.columns
poly15_data['price'] = sales_set_1['price']

lm = linear_model.LinearRegression()
model15 = lm.fit(poly15_data[features], poly15_data['price'])
print(model15.coef_, model15.intercept_)

plt.plot(poly15_data['power_1'],poly15_data['price'],'.')
plt.plot(poly15_data['power_1'], model15.predict(poly15_data[features]), ',', color = 'red', alpha = 1)<jupyter_output>[ 3.05794168e-90 -1.69394968e-49  2.35919957e-55  1.21888065e-88
  3.23082736e-74  1.10358333e-70  8.37724029e-67  6.23411957e-63
  4.49156442e-59  3.06938763e-55  1.91749300e-51  1.01335180e-47
  3.62176959e-44 -5.63501661e-48  2.18641116e-52] 539058.8186680083<jupyter_text>Selecting a Polynomial DegreeNext you should write a loop that does the following:* For degree in [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] (to get this in python type range(1, 15+1))    * Build an SFrame of polynomial data of train_data['sqft_living'] at the current degree    * hint: my_features = poly_data.column_names() gives you a list e.g. ['power_1', 'power_2', 'power_3'] which you might find useful for turicreate.linear_regression.create( features = my_features)    * Add train_data['price'] to the polynomial SFrame    * Learn a polynomial regression model to sqft vs price with that degree on TRAIN data    * Compute the RSS on VALIDATION data (here you will want to use .predict()) for that degree and you will need to make a polynmial SFrame using validation data.* Report which degree had the lowest RSS on validation data (remember python indexes from 0)(Note you can turn off the print out of linear_regression.create() with verbose = False)<jupyter_code>sales_train = pd.read_csv('wk3_kc_house_train_data.csv')
sales_valid = pd.read_csv('wk3_kc_house_valid_data.csv')
sales_test = pd.read_csv('wk3_kc_house_test_data.csv')
RSS_result = []
for degree in range(1,16):
    train_data = polynomial(sales_train['sqft_living'], degree)
    valid_data = polynomial(sales_valid['sqft_living'], degree)
    features = train_data.columns
    train_data['price'] = sales_train['price']
    lm = linear_model.LinearRegression() 
    train_model = lm.fit(train_data[features], train_data['price'])
    predictions = train_model.predict(valid_data[features])
    RSS = np.sum((sales_valid['price'] - predictions)**2)
    RSS_result.append(RSS)
    print('RSS of degree', str(degree), 'model is:', RSS)
RSS_all = pd.DataFrame({'degree': range(1,16), 'RSS': RSS_result})
RSS_all[RSS_all['RSS'] == RSS_all['RSS'].min()]['degree']<jupyter_output>5    6
Name: degree, dtype: int64<jupyter_text>Compute RSS on test data using the selected model<jupyter_code>test_data = polynomial(sales_test['sqft_living'], 6)
train_data = polynomial(sales_train['sqft_living'], 6)

features = train_data.columns # get the name of the features
train_data['price'] = sales_train['price'] # add price to the data since it's the target
lm = linear_model.LinearRegression() 
train_model = lm.fit(train_data[features], train_data['price'])

predictions = train_model.predict(test_data[features])
RSS_test = np.sum((sales_test['price'] - predictions)**2)
print('RSS of test data is:', '{0:.3f}'.format(RSS_test))<jupyter_output>RSS of test data is: 135225113024663.828
{'fraction_non_alphanumeric': 0.13454689354966362, 'fraction_numerical': 0.06120564569318032, 'mean_word_length': 5.9178832116788325, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '536191', 'license': 'MIT', 'n_tokens_mistral': 2931, 'n_tokens_neox': 2656, 'n_words': 600, 'path': 'week3/HW3.ipynb', 'repo_name': 'jessie-sun927/UW-Machine-Learning-Regression', 'size': '7581'}