<jupyter_start><jupyter_text>Using SAS DLPy to Create a Neural Network Model for Regression Using Tabular Data   You can use SAS DLPy to  easily create and train a variety of basic task-oriented deep learning models. This notebook example shows how you can use SAS DLPy to build a fully-connected neural network regression model using tabular data. The model in this notebook uses tables of numeric values to perform a regression task: predict the price of a California home, based on eight housing attributes such as house age, location, average number of bedrooms, population, and so on. The clean tabular data used to create the model is the [Scikit-Learn California Housing data set](https://scikit-learn.org/stable/datasets/real_world.htmlcalifornia-housing-dataset). The California housing data is a real world data set, a 20,640-observation table of 8 numeric, predictive attributes and the target. The target variable is the median house value for California districts. The scikit-Learn data set was derived from the public 1990 U.S. census, using one row per census block group. (A block group is the smallest geographical unit for which the U.S. Census Bureau publishes sample data). The public data set was obtained from [Carnegie Mellon's StatLib repository](http://lib.stat.cmu.edu/datasets/). More technical information about the composition and use of the Scikit-learn California housing data [is available here](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html).The learning objective of this notebook is to understand how to use SAS DLPy to create a fully-connected neural network model designed to perform a tabular data regression task, and how to use model performance plots to assess the model's predictive robustness. The notebook follows these steps: - Configure SAS DLPy and SAS CAS environments for this task.- Upload the California housing data to the SAS CAS server.- Create train and test data sets in SAS CAS.- Use SAS DLPy to build a fully-connected neural network regression model.- Train the fully-connected neural network model.- Evaluate the model training loss statistics.- Use the trained model to score the test data set.- Evaluate the scored test data result statistics.This tutorial example assumes that you have SAS DLPy API installed, have an active SAS CAS server, and have installed the common Python utilities used in the code (numpy, sklearn, matplotlib, Pandas).  Table of Contents - [Prepare Resources and Configure Environment for Modeling](getReady)    - [Import Required Python and SAS DLPy Utilities](importPythonDLPy)- [Configure SAS SWAT and Launch SAS CAS](launchCAS)- [Load the Example Data](loadData)    - [Partition into Train and Test Data](partitionData)- [Use DLPy to Create a Neural Network Model](createModel)- [Train the Neural Network Regression Model](trainModel)- [Evaluate the Training Loss](Plots)        - [Mean Squared Error](MSE)    - [Mean Absolute Error](MAE)    - [Explained Variance Score](explained)- [Score the Test Data](scoreModel)    - [Mean Squared Error](MSE2)    - [Mean Absolute Error](MAE2)    - [Explained Variance Score](explained2)- [Summary](summary)      Prepare Resources and Configure Computing Environment for ModelingUse this section to organize all of the resources you need and to configure your local computing environment in advance. This enables you to follow along with the example notebook modeling operations without interruption.   Import Required Python and SAS DLPy UtilitiesImport the Python and SAS DLPy utilities required for the model task. They include the scikit-Learn utility, the pandas data analysis library, the numPy scientific computing library, and the matplotlib plotting utility.<jupyter_code># Pandas data analysis utility
import pandas as pd

# NumPy scientific computing utility
import numpy as np

# Matplotlib plotting utility
import matplotlib.pylab as plt

# SAS DLPy model utilities
from dlpy import Model, Sequential
from dlpy.model import Optimizer, AdamSolver
from dlpy.layers import * 
from dlpy.splitting import two_way_split
from dlpy.metrics import *

# Display output plots in notebook cells
%matplotlib inline

# Import scikit-Learn utility
import sklearn

# Import SAS SWAT
import swat
from swat import *

# SWAT data message handler
import swat.cas.datamsghandlers as dmh

# SAS DLPy utility modules
from dlpy import Model, Sequential
from dlpy.model import Optimizer, AdamSolver
from dlpy.layers import * 
from dlpy.splitting import two_way_split
from dlpy.metrics import explained_variance_score, mean_squared_error, mean_absolute_error
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Configure SAS SWAT and Launch SAS CASThe following code configures SAS SWAT and launches SAS CAS. SWAT is a Python interface to SAS CAS that enables you to load data into memory and apply CAS actions to the data.Note: For more information about starting a CAS session with the SWAT package, see https://sassoftware.github.io/python-swat/getting-started.html.<jupyter_code># Configure CAS session for Analytics
s = CAS('your-host-name.unx.company-name.com', 5570)<jupyter_output><empty_output><jupyter_text>Load the Example DataUse the code block below to import the scikit-Learn real-world California housing diagnostic data set, create pandas predictor and target dataframes, and then merge the two dataframes. Browse the concatenated pandas table before loading it into SAS CAS.<jupyter_code># Import California Housing data into SAS CAS
# and save the CAS table as california_housing.
from sklearn.datasets import fetch_california_housing
california_housing = fetch_california_housing()

# Put the california_housing contents into two pandas dataframes: 

# One dataframe for the predictors (ch_x_df). 
ch_x_df = pd.DataFrame(california_housing['data'], 
                       columns=california_housing['feature_names']
                       )

# One dataframe for the target (ch_y_df). 
ch_y_df = pd.DataFrame(california_housing['target'], 
                       columns=['house_value']
                       )

# Concatenate the predictor and target dataframes 
# into one dataframe with 8 feature columns and 
# 1 target column.
ch_df = pd.concat([ch_x_df, ch_y_df], axis=1)
# Browse the concatenated pandas housing data.
# Display column headings and 11 rows from the table.
ch_df.loc[:10]<jupyter_output><empty_output><jupyter_text>In the table above, the rightmost column `house_value` represents the median house value for the observation's census block group. Use the `upload_frame()` function with the concatenated table `ch_df` to load the pandas dataframe to the server. The CAS table `ch_df` is loaded and saved in Python as `tbl`.<jupyter_code>tbl = s.upload_frame(ch_df, 
                     casout=dict(name='ch_df', 
                                 replace=True
                                 )
                     );<jupyter_output>NOTE: Cloud Analytic Services made the uploaded file available as table CH_DF in caslib CASUSER(UserID).
NOTE: The table CH_DF has been created in caslib CASUSER(UserID) from binary data uploaded to Cloud Analytic Services.<jupyter_text>Partition into Train and Test DataSplit the loaded California housing table `tbl` into train and test partitions. Use the DLPy `two_way_split` function to perform random server-side sampling and partition the data into `train_tbl` and `test_tbl`. Server-side partitioning is preferable to client-side partitioning when the data is large, or when the data is already on the server. The code below partitions 80% of `tbl` into `train_tbl`, and the remaining 20% is partitioned as `test_tbl`. The `seed` value is specified to ensure repeatability of example computation results.<jupyter_code># Partition tbl into train and test data sets
train_tbl, test_tbl = two_way_split(tbl, 
                                    # 20% test partition 
                                    test_rate=20, 
                                    # random seed value
                                    seed=123, 
                                    # use random sampling 
                                    stratify=False, 
                                    # using tabular numeric data,
                                    # not an image table 
                                    im_table=False)
# Display five rows from the 
# newly created train table.
train_tbl.head()

# Display five rows from the 
# newly created test table.
test_tbl.head()<jupyter_output><empty_output><jupyter_text>Use SAS DLPy to Create a Fully-Connected Neural Network ModelUse SAS DLPy to create a fully-connected neural network regression model with the following architecture:  - One input layer - Two dense layers - 30 neurons in first dense layer  - 10 neurons in the second dense layer - `relu` activation layer - `identity` output layer activation function - `normal error (squared loss)` output layer loss function Save the newly created neural network regression model in Python as `model1`, and save the model table in SAS CAS as `Simple_DNN1`.<jupyter_code># Create fully-connected deep neural network 'model1'
model1 = Sequential(s, 
                    model_table='Simple_DNN1'
                    )

# Input layer
model1.add(InputLayer(std='STD')
           )

# Dense Layer
model1.add(Dense(30, act='relu')
           )
model1.add(Dense(10, act='relu')
          )

# Output layer
model1.add(OutputLayer(act='identity', 
                       error='normal'
                       )
           )<jupyter_output>NOTE: Input layer added.
NOTE: Fully-connected layer added.
NOTE: Fully-connected layer added.
NOTE: Output layer added.
NOTE: Model compiled successfully.<jupyter_text>As an additional exercise, uncomment the bottom line of the code below, and run the cell to generate a DAG of `model1`.<jupyter_code># Generate a DAG for the neural 
# network regression model.

# model1.plot_network()<jupyter_output><empty_output><jupyter_text>The generated DAG for the simple neural network regression model should resemble the following:![SimpleNeuralNetworkRegression.PNG](attachment:SimpleNeuralNetworkRegression.PNG)  Train the Neural Network Regression ModelNow use `fit()` with `model1` to train the fully-connected neural network model using a scheduled training approach with the following hyperparameter specifications:- `AdamSolver` optimizer to show a scheduled training approach using polynomial decay.- `learning_rate_policy='poly'` to specify the polynomial learning rate decreasing policy.- `power=1`, for the learning rate function $initial\_learning\_rate*(1 - \frac{epoch}{max\_epochs})^{power}$.- `log_level=2` to view the training log- `seed=1234` to support model determinism and repeatable results.Save the trained model table in Python as `result`.<jupyter_code># Specify optimizer settings for scheduled training approach
# These optimizer settings are called in the fit() block that 
# follows.
optimizer = Optimizer(algorithm=AdamSolver(learning_rate=0.02, 
                                           learning_rate_policy='poly', 
                                           power=1
                                           ),
                      mini_batch_size=4, 
                      seed=1234, 
                      max_epochs=50, 
                      log_level=2
                      )

# Train the model model1 using 
# data train_tbl, and save the  
# trained output table as 'result' 
result = model1.fit(train_tbl, 
                    # predictor columns
                    inputs=ch_x_df.columns.tolist(), 
                    # target column
                    target='house_value', 
                    # optimizer settings
                    optimizer=optimizer
                    )<jupyter_output>NOTE: Training from scratch.
NOTE:  Synchronous mode is enabled.
NOTE:  The total number of parameters is 591.
NOTE:  The approximate memory cost is 1.00 MB.
NOTE:  Loading weights cost       0.00 (s).
NOTE:  Initializing each layer cost       0.02 (s).
NOTE:  The total number of threads on each worker is 32.
NOTE:  The maximum mini-batch size per thread on each worker is 4.
NOTE:  The maximum mini-batch size across all workers for the synchronous mode is 128.
NOTE:  Target variable: house_value
NOTE:  Number of input variables:     8
NOTE:  Number of numeric input variables:      8
NOTE:  Epoch Learning Rate        Loss  Fit Error   Time(s)
NOTE:  0          0.02          0.3797     0.7593     0.04
NOTE:  1        0.0196          0.2016     0.4032     0.03
NOTE:  2        0.0192          0.1899     0.3797     0.03
NOTE:  3        0.0188          0.1726     0.3452     0.03
NOTE:  4        0.0184          0.1735      0.347     0.03
....  ...       ......          ......     ......     .[...]<jupyter_text>Use the matplotlib function `plot_training_history()` with `model1` to display a graph of the model's training over 50 epochs:<jupyter_code># Plot the training history of 'model1' to
# display the changing model loss and fit 
# error values over 50 epochs.
model1.plot_training_history()<jupyter_output><empty_output><jupyter_text>Score the Training Data to Evaluate Training LossAfter training for the model `model1`, let us examine the training losses. Use the `predict()` function for `model1` with `train_tbl` to score the training data for model fit evaluation. Save the scored CAS data in `model1.valid_res_tbl` in Python as `fitted_table`.<jupyter_code># Score the model1 train table train_tbl
# Save to Python as fitted_result
fitted_result = model1.predict(train_tbl)

# Save the CAS table with scored train 
# results as fitted_table.
fitted_table = model1.valid_res_tbl

# Display the ScoreInfo summary data  
# for the scored train data set.
print(fitted_result['ScoreInfo'])<jupyter_output>Descr         Value
0  Number of Observations Read         16512
1  Number of Observations Used         16512
2           Mean Squared Error      0.274608
3                   Loss Error      0.137304<jupyter_text>Use the `head()` function to show the first five rows of the scored train table `fitted_table`.<jupyter_code># Display the first five rows of
# the scored train data set.
fitted_table.head()<jupyter_output><empty_output><jupyter_text>Mean Squared ErrorWe can use the scored train data set to evaluate the model performance metrics.  For example, we can calculate the MSE. Mean squared error is calculated as the average squared difference between the actual value and the estimated (scored) values. The MSE statistic measures the quality of a predictive estimator.Use the SAS DLPy `mean_squared_error()` function with the table `fitted_table` to compare the MSE between the ground truth `house_value` and the trained model's predicted `P_house_value`. MSE values closer to 0 are preferred.<jupyter_code># Evaluate the MSE using 
# dlpy.metrics.mean_squared_error
mse = mean_squared_error(fitted_table['house_value'], 
                         fitted_table['P_house_value']
                         )
print('The Mean Squared Error for the scored train data is {:.6f}.'.format(mse))<jupyter_output>The Mean Squared Error for the scored train data is 0.274608.<jupyter_text>Mean Absolute ErrorWe can also use the scored data set to calculate another model accuracy measure, the MAE (Mean Absolute Error) statistic. The MAE is the mean of all absolute errors.  Use the SAS DLPy `mean_absolute_error()` function with the table `fitted_table` to compare the MAE between the ground truth `house_value` and the trained model's predicted `P_house_value`. MAE values closer to 0 are preferred.<jupyter_code># Evaluate the MAE using 
# dlpy.metrics.mean_absolute_error
mae = mean_absolute_error(fitted_table['house_value'], 
                          fitted_table['P_house_value']
                          )
print('The Mean Absolute Error for the scored train data is {:.6f}.'.format(mae))<jupyter_output>The Mean Absolute Error for the scored train data is 0.360361.<jupyter_text>Explained Variance ScoreFinally, we can use the scored data and the explained variance score to measure the predictive model accuracy. Explained variance indicates the discrepancy between the ground truth and the predicted values. The explained variance score statistic describes the part of the model's total variance that can be explained by factors that are present, as opposed to differences due to error variance.  Values closer to 0 are bad and values closest to 1 are desirable. Use the SAS DLPy `explained_variance_score()` function with the table `fitted_table` to examine the explained variance between the ground truth `house_value` and the trained model's predicted `P_house_value`. Explained variance percentages close to 0 are undesirable, percentages closest to 100 are desirable.<jupyter_code># Evaluate the explained variance score 
# using dlpy.metrics.explained_variance_score
evs = explained_variance_score(fitted_table['house_value'], 
                               fitted_table['P_house_value']
                               )
print('{:.2f}% of the target variable variance in the scored train data '.format(evs*100))
print('can be explained by the model.')<jupyter_output>79.53% of the target variable variance in the scored train data 
can be explained by the model.<jupyter_text>Score the Test Data Using Trained Neural Network ModelNow use the `predict()` function for `model1` with `test_tbl` to score the test data. Save the scored data in `model1.valid_res_tbl` in Python as `test_result_table`.<jupyter_code># Use model1 to score the test data test_tbl.
test_result = model1.predict(test_tbl)

# Save the scored data table as test_result_table.
test_result_table  = model1.valid_res_tbl

# Display the ScoreInfo summary data  
# for the scored test data set.
print(test_result['ScoreInfo'])
# Display 5 sample rows from the scored
# test data set result table.
test_result_table.head()<jupyter_output><empty_output><jupyter_text>Mean Squared ErrorNow we can use the scored test data set to evaluate the model performance metrics.  For example, we can calculate the MSE (Mean Squared Error), the average squared difference between the estimated (scored) values and the ground truth value. The MSE statistic measures the quality of a model predictor. Use the SAS DLPy `mean_squared_error()` function with the table `test_result_table` to compare the MSE between the ground truth `house_value` and the trained model's predicted `P_house_value`. MSE values closer to 0 are desirable.<jupyter_code># Evaluate the MSE using 
# dlpy.metrics.mean_squared_error
mse = mean_squared_error(test_result_table['house_value'], 
                         test_result_table['P_house_value']
                         )
print('The Mean Squared Error for the scored test data is {:.6f}.'.format(mse))<jupyter_output>The Mean Squared Error for the scored test data is 0.275125.<jupyter_text>Mean Absolute ErrorWe can also use the scored test data set to calculate the MAE (Mean Absolute Error) statistic. The MAE is the mean of all absolute errors in the scored data. A MAE score close to 0 is desirable.  Use the SAS DLPy `mean_absolute_error()` function with the table `test_result_table` to compare the MAE between the ground truth `house_value` and the trained model's predicted `P_house_value`. MAE values closer to 0 are desirable.<jupyter_code># Evaluate the MAE using dlpy.metrics.mean_absolute_error
mae = mean_absolute_error(test_result_table['house_value'], 
                          test_result_table['P_house_value']
                          )
print('The Mean Absolute Error for the scored test data is {:.6f}.'.format(mae))<jupyter_output>The Mean Absolute Error for the scored test data is 0.364937.<jupyter_text>Explained Variance ScoreFinally, we can use the scored data and the explained variance score to measure the predictive model accuracy. Explained variance examines the discrepancy between the ground truth and the predicted values. The explained variance score describes the portion of the model's total variance that can be explained by existing model factors, as opposed to differences due to error variance.  Explained variance percentage values close to 0 are undesirable, percentage values close to 100% are desirable. Use the SAS DLPy `explained_variance_score()` function with the test results table to analyze explained variance for predicted house values versus ground truth house values. The greater the percentage of explained variance, the less randomness there is in the model predictions.<jupyter_code># Evaluate the model's explained variance  
# using dlpy.metrics.explained_variance_score
evs = explained_variance_score(test_result_table['house_value'], 
                               test_result_table['P_house_value']
                               )
print('{:.2f}% of the target variable variance in the scored test data'.format(evs*100))
print('can be explained by the model.')<jupyter_output>78.76% of the target variable variance in the scored test data
can be explained by the model.
{'fraction_non_alphanumeric': 0.06081466008355488, 'fraction_numerical': 0.0141473604253703, 'mean_word_length': 3.5388924800689505, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '550948', 'license': 'Apache-2.0', 'n_tokens_mistral': 5835, 'n_tokens_neox': 5349, 'n_words': 2413, 'path': 'examples/tabular_data_analysis/Regression_with_Fully_Connected_Neural_Network.ipynb', 'repo_name': 'ChipRobie/python-dlpy', 'size': '21064'}