<jupyter_start><jupyter_text>Quick demo of using sklearn for regressionThis demo covers:* KNeighborsRegressor  : https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsRegressor.html* DecisionTreeRegressor : https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.htmlsklearn.tree.DecisionTreeRegressorYou can see below uses of 'fit' and 'predict' for a 1-dim input to 1-dim output regression task based on the "abalone" dataset from UCI (also used in HW1).FYI, the first part of the notebook has a brief demo of how arrays may share or not share data after assignment. This is a common GOTCHA for early Python NumPy users.<jupyter_code># imports and setup
import numpy as np

import seaborn as sns
import matplotlib
import matplotlib.pyplot as plt

import math

import pandas as pd

%matplotlib inline
np.set_printoptions(suppress=True, precision=2)
pd.set_option('precision', 2) # number precision for pandas
plt.style.use('seaborn') # pretty matplotlib plots<jupyter_output><empty_output><jupyter_text>Load data from csv file<jupyter_code>with open('datasets/data_abalone/x_train.csv', 'r') as f:
    for line in f.readlines()[:5]:
        print(line.strip())
x_df = pd.read_csv('datasets/data_abalone/x_train.csv')
x_NF = x_df.values.copy()<jupyter_output><empty_output><jupyter_text>Assigning "arrB = arrA" does NOT copy underlying data FYI So does slicing like "arrB = arrA[:10]" Thus, any modifications to arrB will impact arrA as well!<jupyter_code>arrA = x_NF.copy()
arrA.shape
arrA[:5]
arrB = arrA # NOT A COPY!
arrB[0] = 1. # assigns first row of data in BOTH arrA and arrB
arrB[:5]
arrA[:5]<jupyter_output><empty_output><jupyter_text>Assigning "arrB = arrA.copy()" DOES copy underlying dataThus, any modifications to arrB will NOT impact arrA at all.<jupyter_code>arrA = x_NF.copy()
arrA[:5]
arrB = arrA.copy() # Make a complete copy of array with separately allocated data in memory
arrB[0] = 1.0 # assign first row to all ones
arrA[:5]
arrB[:5]<jupyter_output><empty_output><jupyter_text>Let's zoom in on one data analysis taskRegression of 'rings' given 'length' of abalone<jupyter_code>x_df = pd.read_csv('datasets/data_abalone/x_train.csv')
x_NF = x_df.values.copy()
for col in x_df.columns:
    print(col)<jupyter_output>is_male
length_mm
diam_mm
height_mm
whole_weight_g
shucked_weight_g
viscera_weight_g
shell_weight_g<jupyter_text>Keep only specific column related to length<jupyter_code>xlength_N1 = x_NF[:, 1:2]
xlength_N1.shape<jupyter_output><empty_output><jupyter_text>Load in 'rings' data from y_train.csv<jupyter_code>y_df = pd.read_csv('datasets/data_abalone/y_train.csv')
y_N = y_df.values[:,0]
y_N.shape<jupyter_output><empty_output><jupyter_text>1-Nearest Neighbor Regression: length -> rings  Fit<jupyter_code>import sklearn.neighbors
k1nn_regr = sklearn.neighbors.KNeighborsRegressor(
    n_neighbors=1, algorithm='brute', metric='euclidean')
k1nn_regr.fit(xlength_N1, y_N)<jupyter_output><empty_output><jupyter_text>Predict with K=1 neighborsShow the predictions at G=100 evenly spaced example lengthsRemember, calls to predict always need to have shape (n_examples x n_features)Since we have one feature, we use [:,np.newaxis] to expand a G-shape into a (G,1) shape array<jupyter_code>x_grid_G = np.linspace(xlength_N1.min(), xlength_N1.max(), 100)
yhat_k1nn_G = k1nn_regr.predict(x_grid_G[:,np.newaxis])
plt.plot(xlength_N1, y_N, 'k.', alpha=0.2)
plt.plot(x_grid_G, yhat_k1nn_G, 'b.-', linewidth=3);
plt.xlabel('length (mm)'); plt.ylabel('ring count');<jupyter_output><empty_output><jupyter_text>5-Nearest Neighbor Regression: length -> rings  Fit<jupyter_code>k5nn_regr = sklearn.neighbors.KNeighborsRegressor(
    n_neighbors=5, algorithm='brute', metric='euclidean')
k5nn_regr.fit(xlength_N1, y_N)<jupyter_output><empty_output><jupyter_text>PredictShow the predictions at G=100 evenly spaced example lengthsRemember, calls to predict always need to have shape (n_examples x n_features)Since we have one feature, we use [:,np.newaxis] to expand a G-shape into a (G,1) shape array<jupyter_code>x_grid_G = np.linspace(xlength_N1.min(), xlength_N1.max(), 100)
yhat_k5nn_G = k5nn_regr.predict(x_grid_G[:,np.newaxis])<jupyter_output><empty_output><jupyter_text>Visualize predictions and training data for K=5 NN<jupyter_code>plt.plot(xlength_N1, y_N, 'k.', alpha=0.2)
plt.plot(x_grid_G, yhat_k5nn_G, 'b.-', linewidth=3);
plt.xlabel('length (mm)'); plt.ylabel('ring count');
plt.title('5-Nearest Neighbor Regression');<jupyter_output><empty_output><jupyter_text>Decision Tree Regression: length -> rings  Fit<jupyter_code>import sklearn.tree
tree_regr = sklearn.tree.DecisionTreeRegressor(
    random_state=42, min_samples_leaf=500)
tree_regr.fit(xlength_N1, y_N)<jupyter_output><empty_output><jupyter_text>PredictShow the predictions at G=100 evenly spaced example lengthsRemember, calls to predict always need to have shape (n_examples x n_features)Since we have one feature, we use [:,np.newaxis] to expand a G-shape into a (G,1) shape array<jupyter_code>x_grid_G = np.linspace(xlength_N1.min(), xlength_N1.max(), 100)
yhat_tree_G = tree_regr.predict(x_grid_G[:,np.newaxis])
plt.plot(xlength_N1, y_N, 'k.', alpha=0.2)
plt.plot(x_grid_G, yhat_tree_G, 'b.-', linewidth=3);
plt.xlabel('length (mm)'); plt.ylabel('ring count');
plt.title('DTree Regression with min_samples_leaf=100');<jupyter_output><empty_output><jupyter_text>Visualizing decision trees with 'pydotplus'This section requires a local install of `GraphViz` (https://www.graphviz.org/) and the `pydotplus` Python package.You can simply do:* `conda install -c anaconda graphviz`* `conda install pydotplus`inside your comp135_env environment.<jupyter_code>from sklearn.externals.six import StringIO  
from IPython.display import Image  
import pydotplus
# Make a variable to store some string info to
dot_data = StringIO()
# Call 'export_graphviz' provided by sklearn
sklearn.tree.export_graphviz(
    tree_regr, out_file=dot_data,
    feature_names=['length_mm'],
    precision=2,
    filled=True, rounded=True,
    special_characters=True)
graph_viz_obj = pydotplus.graph_from_dot_data(dot_data.getvalue())
Image(graph_viz_obj.create_png())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11207589644637402, 'fraction_numerical': 0.015275767808329314, 'mean_word_length': 6.943805874840358, 'pattern_counts': {'":': 0, '<': 60, '<?xml version=': 0, '>': 63, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '664393', 'license': 'MIT', 'n_tokens_mistral': 2374, 'n_tokens_neox': 2277, 'n_words': 535, 'path': 'labs/RegressionDemo.ipynb', 'repo_name': 'tufts-ml-courses/comp135-19s-assignments', 'size': '6219'}