<jupyter_start><jupyter_text>1. Download the housing dataset referenced in the textbook on page 279, https://archive.ics.uci.edu/ml/machine-learning-databases/housing/<jupyter_code>df = pd.read_csv('https://raw.githubusercontent.com/rasbt/python-machine-learning-book-3rd-edition/'
                 'master/ch10/housing.data.txt',
                 header=None,sep='\s+')

df.columns = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 
              'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']
df.head()<jupyter_output><empty_output><jupyter_text>2. Pick a number of features that you think may be correlated and plot pairs of them to confirm that they are correlated.Plotting features that are possibly correlatedFeatures chosen are CRIM, ZN, RM, AGE, TAX, PTRATIO AND LSTAT<jupyter_code>cols = ['CRIM', 'ZN', 'RM', 'AGE', 'TAX', 'PTRATIO', 'LSTAT']
scatterplotmatrix(df[cols].values, figsize=(10, 8), names=cols, alpha=0.5)
plt.tight_layout()
plt.show()
cm = np.corrcoef(df[cols].values.T)
hm = heatmap(cm, row_names=cols, column_names=cols)
plt.show()<jupyter_output><empty_output><jupyter_text>As seen from the heatmap above, LSTAT and AGE showed the greatest correlation, while LSTAT and RM had the lowest correlation.  3. Pick one feature that you think can be predicted by the other features in the dataset. The feature to be predicted needs to have numerical values.Based on the heatmap obtained in the previous section, the feature to be predicted was chosen as LSTAT. Switching the positions of columns "LSTAT" and "MEDV" makes later steps easier<jupyter_code>columns_titles = ["CRIM", "ZN", "INDUS", "CHAS", "NOX", "RM", "AGE", "DIS", "RAD", 
              "TAX", "PTRATIO", "B", "MEDV", "LSTAT"]
df=df.reindex(columns=columns_titles)
df<jupyter_output><empty_output><jupyter_text>4. Separate the data into training, validation, and test sets.Spliting the dataset into training and testing sets; no validation set was created because it wasn't involved in any of the later steps.<jupyter_code>X = df.iloc[:, :-1].values
y = df['LSTAT'].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)<jupyter_output><empty_output><jupyter_text>5. Apply various Scikit Learn regression methods to the data.Using linear regression mode:<jupyter_code>slr = LinearRegression()
slr.fit(X_train, y_train)
y_train_pred = slr.predict(X_train)
y_test_pred = slr.predict(X_test)
print(slr.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>5. Apply various Scikit Learn regression methods to the data.Using lasso mode:<jupyter_code>lasso = Lasso(alpha=0.1)
lasso.fit(X_train, y_train)
y_train_pred = lasso.predict(X_train)
y_test_pred = lasso.predict(X_test)
print(lasso.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>5. Apply various Scikit Learn regression methods to the data.Using ridge mode:<jupyter_code>ridge = Ridge(alpha=1.0)
ridge.fit(X_train, y_train)
y_train_pred = ridge.predict(X_train)
y_test_pred = ridge.predict(X_test)
print(ridge.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>5. Apply various Scikit Learn regression methods to the data.Using elastic net mode:<jupyter_code>elanet = ElasticNet(alpha=1.0, l1_ratio=0.5)
elanet.fit(X_train, y_train)
y_train_pred = elanet.predict(X_train)
y_test_pred = elanet.predict(X_test)
print(elanet.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>7. Repeat steps 3. to 6. for a second feature.Picking a second featureThe second feature chosen to be predicted was CRIM.Switching the positions of "CRIM" and "MEDV" columns make later steps easier.<jupyter_code>columns_titles = ["ZN", "INDUS", "CHAS", "NOX", "RM", "AGE", "DIS", "RAD", 
              "TAX", "PTRATIO", "B", "LSTAT", "MEDV", "CRIM"]
df=df.reindex(columns=columns_titles)
df<jupyter_output><empty_output><jupyter_text>Spliting the dataset into training and testing sets; no validation set was created because it wasn't involved in any of the later steps.<jupyter_code>X = df.iloc[:, :-1].values
y = df['CRIM'].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)<jupyter_output><empty_output><jupyter_text>Using linear regression mode:<jupyter_code>slr = LinearRegression()
slr.fit(X_train, y_train)
y_train_pred = slr.predict(X_train)
y_test_pred = slr.predict(X_test)
print(slr.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>Using lasso mode:<jupyter_code>lasso = Lasso(alpha=0.1)
lasso.fit(X_train, y_train)
y_train_pred = lasso.predict(X_train)
y_test_pred = lasso.predict(X_test)
print(lasso.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>Using ridge mode:<jupyter_code>ridge = Ridge(alpha=1.0)
ridge.fit(X_train, y_train)
y_train_pred = ridge.predict(X_train)
y_test_pred = ridge.predict(X_test)
print(ridge.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output><jupyter_text>Using elastic net mode:<jupyter_code>elanet = ElasticNet(alpha=1.0, l1_ratio=0.5)
elanet.fit(X_train, y_train)
y_train_pred = elanet.predict(X_train)
y_test_pred = elanet.predict(X_test)
print(elanet.coef_)
print('MSE train: %.3f, test: %.3f' % (mean_squared_error(y_train, y_train_pred),
        mean_squared_error(y_test, y_test_pred)))
print('R^2 train: %.3f, test: %.3f' % (r2_score(y_train, y_train_pred),
        r2_score(y_test, y_test_pred)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1410752688172043, 'fraction_numerical': 0.014910394265232975, 'mean_word_length': 6.082233502538071, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '662040', 'license': 'MIT', 'n_tokens_mistral': 2770, 'n_tokens_neox': 2688, 'n_words': 572, 'path': 'assignment8.ipynb', 'repo_name': 'yuxuancs/Machine_Learning', 'size': '6975'}