<jupyter_start><jupyter_text>Tutorial developed from Medium article["Introduction to Supervised Learning with Python: Scikit-Learn Tutorial](https://medium.com/@mwitiderrick/introduction-to-supervised-learning-with-python-scikit-learn-tutorial-966457dd33b9)  Loading the DataUsing USA Housing dataset available at[Kaggle](https://www.kaggle.com/vedavyasv/usa-housing/dataUSA_Housing.csv).  Download the data set, unzipit, and put it into your data directory (or change path appropriately to readthe csv file).<jupyter_code># load the data
df = pd.read_csv('../../data/USA_Housing.csv')
display(df.info())
display(df.columns)
display(df.describe())
display(df.head())
sns.pairplot(df);
sns.heatmap(df.corr(), annot=True);<jupyter_output><empty_output><jupyter_text>Selection of Features- drop Price, because that is actually the target we wish to predict or model.- drop Address, because this is a textual field.  If we want to use this  information in a machine model, we will need to transform it into  numerical information (we could turn it into a gps coordinate, for example,  but would this really give us any better information than the existing  features?)<jupyter_code># selection of features
X = df.drop(['Price', 'Address'], axis=1)
display(X.columns)
# the targets for our models will be in y
y = df['Price']
display(y.describe())<jupyter_output><empty_output><jupyter_text>Create Training and Test SetsAn example of using utility functions from `sklearn` for a simple train/testsplit of the data.<jupyter_code>from sklearn.model_selection import train_test_split

# use 30% for the test data, and 70% for the training
# set random_state seed, so we can recreate this exact same train/test
# split if needed.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

display(X_train.shape)
display(y_train.shape)
display(X_test.shape)
display(y_test.shape)<jupyter_output><empty_output><jupyter_text>Create and Train a Linear Regression ModelAn example of creating a simple linear regression model using the `sklearn`framework.  All basic models in `sklearn` are objects that have the following memberfunctions:- fit() Train model with input data (X) to fit the training data (y)- predict() Given a trained model, predict the target value on (possibly new  and unseen) data.<jupyter_code>from sklearn.linear_model import LinearRegression

# create our model instance object
lm = LinearRegression()

# train the model to predict using the training data
lm.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Linear Regression Model EvaluationEvaluate how well this model performs.The coefficients are the learned theta parameters of the linearregression model.<jupyter_code>display(lm.coef_)

# trick to display the coefficient associated with each feature
# put back into a DataFrame, and specify the column names from X
coeff = pd.DataFrame(lm.coef_, X.columns, columns=['coefficients'])
display(coeff)<jupyter_output><empty_output><jupyter_text>Now we make predictions, using the predict() function.  Here we make predictionsof the held back test data.<jupyter_code>predictions = lm.predict(X_test)

# visualize the predictions, perfect predictions would show as
# a perfectly straight line,, where each predictions exactly matched
# the true value of y_test
sns.jointplot(y_test, predictions);<jupyter_output><empty_output><jupyter_text>As we discussed a bit in previous lectures, residuals are important informationin evaluating the performance of a model.  If the residuals are notnormally distributed, this probably indicates that the model (a linearmodel in this case) is not appropriate for this data.<jupyter_code>sns.distplot( (y_test - predictions), bins=30);<jupyter_output><empty_output><jupyter_text>Use evaluation metrics to evaluate the linear regression model.<jupyter_code>from sklearn import metrics

print('Mean Absolute Error      (MAE): ', metrics.mean_absolute_error(y_test, predictions))
print('Mean Squared Error       (MSE): ', metrics.mean_squared_error(y_test, predictions))
print('Root Mean Squared Error (RMSE): ', np.sqrt(metrics.mean_squared_error(y_test, predictions)))<jupyter_output>Mean Absolute Error      (MAE):  81135.56609336875
Mean Squared Error       (MSE):  10068422551.401083
Root Mean Squared Error (RMSE):  100341.52954485537<jupyter_text>Create and Train a Logistic Regression ModelThe previous dataset was a regression problem (predicting a real valuedprice).  So we will need a different dataset for a logistic regressionexample.  But, as you will see, the framework of creating, fitting andevaluating the model remains the same.The kyphosis dataset is also available from kaggle.  You will need todownload it and unzip it to the correct location once again in orderto use it for the next examples.The kyphosis column in the data set is a binary category (present or absent)that is what we want to model and predict for this data.<jupyter_code># read in the data set
kyphosis = pd.read_csv('../../data/kyphosis.csv')

display(kyphosis.info())
display(kyphosis.columns)
display(kyphosis.describe())
display(kyphosis.head())
# select the features for training
# and do a train/test split
X = kyphosis.drop(['Kyphosis'], axis=1)
y = kyphosis['Kyphosis']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

display(X_train.shape)
display(y_train.shape)
display(X_test.shape)
display(y_test.shape)
# train a logistic regression model
from sklearn.linear_model import LogisticRegression

lr = LogisticRegression()
lr.fit(X_train, y_train)
# after fitting model, use predict to predict the labels of test data
k_pred = lr.predict(X_test)
display(k_pred)
# evaluate the model
from sklearn.metrics import classification_report, confusion_matrix

display(confusion_matrix(y_test, k_pred))
print(classification_report(y_test, k_pred))<jupyter_output>precision    recall  f1-score   support

      absent       0.76      1.00      0.86        19
     present       0.00      0.00      0.00         6

    accuracy                           0.76        25
   macro avg       0.38      0.50      0.43        25
weighted avg       0.58      0.76      0.66        25<jupyter_text>Decision Tree and Random Forest Models<jupyter_code># train and fit a decision tree model
from sklearn.tree import DecisionTreeClassifier

dt = DecisionTreeClassifier()
dt.fit(X_train, y_train)
# evaluate the model
dt_pred = dt.predict(X_test)

display(confusion_matrix(y_test, dt_pred))
print(classification_report(y_test, dt_pred))
# train and fit a random forest
from sklearn.ensemble import RandomForestClassifier

rfc = RandomForestClassifier(n_estimators=100)

rfc.fit(X_train, y_train)
# evaluate the model
rfc_pred = rfc.predict(X_test)

display(confusion_matrix(y_test, rfc_pred))
print(classification_report(y_test, rfc_pred))
import sys
sys.path.append("../../src") # add our class modules to the system PYTHON_PATH

from ml_python_class.custom_funcs import version_information
version_information()<jupyter_output>Module   Versions
--------------------   ------------------------------------------------------------
              numpy:   ['1.18.5']
             pandas:   ['1.0.5']
            seaborn:   ['0.10.1']
{'fraction_non_alphanumeric': 0.09401827748546109, 'fraction_numerical': 0.01883134865688175, 'mean_word_length': 4.746221161495624, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '63594', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 2265, 'n_tokens_neox': 2136, 'n_words': 724, 'path': 'lectures/archive/Lecture-scikit-learn-ml-framework-archive.ipynb', 'repo_name': 'tgrasty/CSCI574-Machine-Learning', 'size': '7222'}