<jupyter_start><jupyter_text>Build and Train Keras LeNet Model, Convert to SAS Deep Learning LeNet Model with DLPyThis example trains a LeNet model in Keras, and then imports the resulting trained model into a SAS Deep Learning model using SAS DLPy.  The test image data is scored by both Keras and DLPy models, and the model results are compared. The results show that the imported and converted SAS Deep Learning model duplicates the results that were produced by the original Keras model. This demonstrates the consistency between the open-source Keras model and the imported and converted SAS Deep Learning model.  For best success before attempting this example in your own environment, consider reading <NAME>'s tutorial blog post on LeNet CNN models: https://www.pyimagesearch.com/2016/08/01/lenet-convolutional-neural-network-in-python. The blog provides a nice summary of LeNet, and this example follows the blog's LeNet model definition.  Important Note: Client and Server DefinitionsSAS Viya and Deep Learning literature and technical documentation often refers to client and server entities.  In this scenario, the client is the computer that runs the Jupyter notebook with the example code.  The server is the computer that is running the SAS Viya server.  These two computers might (or might not) use the same operating system, and might (or might not) have access to a common file system.  This example assumes that the client and server do not use the same operating system, but that they do have access to a common file system.  If the client and server in your environment do not have access to a common file system, you will need to copy or transfer files between client and server twice during this example.  The example in this notebook uses comments in cells to indicate when a given file should be moved from the client to the server.<jupyter_code># define variables to contain path specifications to  
# the client and server machine image root directories

client_image_root = "/client/path-to/image-root/directory" # this is the CLIENT_IMAGE_ROOT directory

server_image_root = "/server/path-to/image-root/directory" # this is the SERVER_IMAGE_ROOT directory<jupyter_output><empty_output><jupyter_text>Import Python Function Packages Here we import and intitialize Python function packages that will be used during the example. The Keras function packages are used to build the LeNet model in Keras before the trained model is imported as a SAS Viya Deep Learning model using SAS DLPy.<jupyter_code>from __future__ import print_function

import sys
import warnings

if not sys.warnoptions:
    warnings.simplefilter("ignore")

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Conv2D, MaxPooling2D, Flatten
from keras.optimizers import RMSprop
import os
from PIL import Image
import numpy as np<jupyter_output>Using TensorFlow backend.<jupyter_text>Extract the Keras MNIST Image Classification DataThis step of the example extracts the MNIST data using Keras utilities. For more information on the MNIST data, see https://keras.io/datasets/mnist-database-of-handwritten-digits. Convenience functions provided by Keras load the data and assign labels to the training and test data.  There should be 60000 training images and 10000 test images.  Each image should be 28 pixels by 28 pixels.<jupyter_code>num_classes = 10

# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

# convert class vectors to binary class matrices - this effectively gives a classification
# label for each image
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

print("Number of training images = " + str(x_train.shape[0]))
print("Number of test images = " + str(x_test.shape[0]))
print("Image size = " + str(x_train.shape[1]) + " x " + str(x_train.shape[2]))<jupyter_output>Number of training images = 60000
Number of test images = 10000
Image size = 28 x 28<jupyter_text>Prepare Test Images Before Loading into SAS CAS The test images for the example are stored in Python numpy arrays. There is not a direct way to convert a numpy array to a SAS CAS table, so the example first stores the test images in an intermediate PNG format. Afterwards, the example performs image processing actions and then loads the stored PNG images into a SAS CAS table.  Note that the action that loads images expects the following directory structure:CLIENT_IMAGE_ROOT/label1CLIENT_IMAGE_ROOT/label2...CLIENT_IMAGE_ROOT/labelNIn this instance, CLIENT_IMAGE_ROOT specifies the name of the root image directory on the client, and "label1", "label2", ..., "labelN" represents the N image classes.  All the images that belong to image class "label1" should reside in the CLIENT_IMAGE_ROOT/label1 directory, all the images that belong to image class "label2" should reside in the CLIENT_IMAGE_ROOT/label2 directory, and so on.The MNIST image data contains 10 object classes. Each class is assigned a label "0", "1", ..., "9".  The example code assigns the test images to 10 separate directories as follows:CLIENT_IMAGE_ROOT/0CLIENT_IMAGE_ROOT/1...CLIENT_IMAGE_ROOT/9Note: This example assumes that if the CLIENT_IMAGE_ROOT directory already exists, then all image files have been previously unpacked and are ready for processing.<jupyter_code># check if working directory exists
if os.path.isdir(client_image_root):
    print("Image directory exists, assuming image files are prepared for loading into CAS table.")
else:
    os.mkdir(client_image_root)
    
    # create subdirectories for each image class
    for ii in range(10):
        new_path = os.path.join(client_image_root,str(ii))
        if not os.path.isdir(new_path):
            os.mkdir(new_path)

    # distribute images to proper directories
    im_index = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    for ii in range(x_test.shape[0]):
        im = Image.fromarray(x_test[ii,])
        idx = np.where(y_test[ii])[0][0]
        file_name = 'im'+str(idx)+'_file'+str(im_index[idx])+'.png'
        new_path = os.path.join(client_image_root,str(idx))
        im.save(os.path.join(new_path,file_name))
        im_index[idx] = im_index[idx] + 1
# If your client/server do not share a common file system, please copy the image files and subdirectories from CLIENT_IMAGE_ROOT to 
# SERVER_IMAGE_ROOT.<jupyter_output><empty_output><jupyter_text>Data Pre-processingThe following code converts the image data to floating point and scales it such that all pixel values fall within the range 0.0 to 1.0.<jupyter_code># Scale the 60,000 train images and the 10,000 test images.
x_train = x_train.reshape(60000, 28, 28, 1)
x_test = x_test.reshape(10000, 28, 28, 1)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255<jupyter_output><empty_output><jupyter_text>Define the Keras LeNet Model Architecture<jupyter_code># Define the Architecture for a Keras LeNet 
# Image Classification Model

# training parameters
batch_size = 128
epochs = 1

# model architecture and activation functions
model = Sequential()
model.add(Conv2D(20, kernel_size=(5, 5), strides=(1, 1), activation='relu', input_shape=(28,28,1), padding="same"))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='same'))
model.add(Conv2D(50, kernel_size=(5, 5), strides=(1, 1), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='same'))
model.add(Flatten())
model.add(Dense(500, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))

# print a model summary
model.summary()<jupyter_output>Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 20)        520       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 20)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 14, 14, 50)        25050     
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 7, 7, 50)          0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 2450)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 500)               1225500   
______________________________________________________[...]<jupyter_text>Provide Trained LeNet Model ParametersAfter you define the Keras LeNet model architecture, you must either train the model from scratch, or use an exisiting trained model parameter file in HDF5 (.h5) format. You can make changes in the example code below according to your choice.If you have not run this example code before, you should use the code block that sets the `do_train` parameter to True. This setting trains the model from scratch, and then saves the weighted results in a generated .h5 file named `lenet.h5`. If you are running subsequent instances of this example (perhaps experimenting with parameter changes) it is not necessary to train the model from scratch each time. Instead, you can use the generated `lenet.h5` file that was created after training the model from scratch. To do this, you should comment out the `do_train = True` code block, and instead use the `do_train = False` code block below. The location of the trained `lenet.h5` parameter file should be specified in `CLIENT_IMAGE_ROOT/lenet.h5`.<jupyter_code># Compile the model using categorical cross entropy as the loss 
# function and RMS propagation as the iterative training optimizer.

model.compile(loss='categorical_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy']
              )

# If training for the first time, set the `do_train` parameter to True.
# This will train the model from scratch, and save the pre-trained weighted 
# model parameters in a file named "Lenet.h5"

# If you have already trained the model from scratch, you can use the 
# saved "Lenet.h5" file to load the model using pre-trained weights, 
# instead of spending processor time to train from scratch.

# For first time training, use the following code:

do_train = True
if do_train:
    history = model.fit(x_train, y_train,
                        batch_size=batch_size,
                        epochs=epochs,
                        verbose=1,
                        validation_data=(x_test, y_test)
                       )
    
    model.save(os.path.join(client_image_root,"lenet.h5"))


# If you have already trained the model from scratch, commment out the 
# previous code block, and un-comment the following code block. The    
# following code sets the `do_train` parameter to False, and uses the  
# generated model weights (Lenet.h5) file for the model:               


# do_train = False
# if do_train:
#    history = model.fit(x_train, y_train,
#    batch_size=batch_size,
#    epochs=epochs,
#    verbose=1,
#    validation_data=(x_test, y_test))
    
#    model.save(os.path.join(client_image_root,"lenet.h5"))

# else:

# Load the previously trained parameter file "lenet.h5" instead
# of training the LeNet model from scratch.

#    model.load_weights(os.path.join(client_image_root,"lenet.h5"))

# At this point you have a trained model network.
# Now use the trained model (whether trained by scratch or using 
# the pre-trained model parameter file) to score the test Data.
    
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])<jupyter_output>Train on 60000 samples, validate on 10000 samples
Epoch 1/1
60000/60000 [==============================] - 35s 584us/step - loss: 0.1499 - accuracy: 0.9542 - val_loss: 0.0371 - val_accuracy: 0.9875
Test loss: 0.037071275619626975
Test accuracy: 0.987500011920929<jupyter_text>Connect to the SAS CAS Server<jupyter_code># Configure matplotlib utility to display output in 
# Jupyter notebook cell
%matplotlib inline

# SAS Scripting Wrapper for Analytic Transfer (SWAT)
# is a module needed to connect to a CAS server.
import swat
from swat import * 

host_name='your-qualified.server-name.com'


# port_number='12345' # use a unique 5-digit port ID
port_number=12345

s = swat.CAS('your-qualified.server-name.com', 12345)<jupyter_output>NOTE: Added action set 'deepLearn'.<jupyter_text>Convert the Keras Model to a SAS Deep Learning ModelTwo operations take place when the SAS DLPy `from_keras_model()` function is called. The first operation automatically translates the Keras model into a SAS Deep Learning model.  You can perform this process manually, but it is not recommended because it is error-prone and time-consuming.  The second operation rearranges and reformats the HDF5 parameter file that was generated by Keras.  Rearranging and reformatting the parameter file enables the SAS Deep Learning action set to correctly parse and read the trained parameters.<jupyter_code>from dlpy import Model
# Convert the Keras model to a SAS Deep Learning model
# Convert Keras model parameters to a format 
# that can be read by SAS DLPy and the SAS 
# Deep Learning tools.

model_name = 'lenet'
model1,use_gpu = Model.from_keras_model(conn=s, 
                                        keras_model=model, 
                                        output_model_table=model_name,
                                        include_weights=True, 
                                        scale=1.0/255.0,
                                        input_weights_file=os.path.join(client_image_root,"lenet.h5")
                                       )<jupyter_output>WARNING: could not find layer conv2d_1_input, in model. Translated model may be inaccurate.
NOTE: Model table is attached successfully!
NOTE: Model is named to "lenet" according to the model name in the table.
NOTE: the model weights has been stored in the following file:
C:\Users\chrobi\python-dlpy-1\examples\keras_model_conversion\lenet_weights.kerasmodel.h5<jupyter_text>NOTE: You should not be concerned about the warning in the example output. In this case, it is expected, because the Keras model input layer did not have a name. The warning simply informs users that a name has been created for the Viya model input layer.  Copy the Reformatted HDF5 FileThis is for convenience, in the event that your client and server share a common file system.  If there is no common file system, then you must perform a copy operation to transfer the HDF5 file from one file system to another.  Remember that deep learning HDF5 files can be quite large, so the data transfer could take a significant amount of time.<jupyter_code>from shutil import copyfile
copyfile('lenet_weights.kerasmodel.h5', os.path.join(client_image_root,"lenet_weights.kerasmodel.h5"))<jupyter_output><empty_output><jupyter_text>Populate the SAS Deep Learning Model with Trained ParametersUp to this point, the SAS Deep Learning model contains parameters that were initialized using the default settings that were supplied when the model was created in the preceding step. Now we will import the trained parameters from the Keras model and install them into the SAS Viya model. The weighted parameter file is called `lenet_weights.kerasmodel.h5` and it should be downloaded and stored in a subdirectory to the server path specfication you stored earlier in the variable `server_image_root`.It is also a good idea at this time to apply labels to the output of the classification layer. The layers should be consistent with the labels that we will shortly apply to the test images. The label file is called `mnist_labels.csv` and it can be downloaded from the [SAS DLPy example folder for Keras examples](https://github.com/sassoftware/python-dlpy/tree/master/examples/keras_model_conversion) on GitHub.<jupyter_code>model1.load_weights(path=server_image_root+"/lenet_weights.kerasmodel.h5",
                    labels=True,
                    label_file_name=os.path.join(os.getcwd(),"mnist_labels.csv"),
                    label_length=1)<jupyter_output>NOTE: Cloud Analytic Services made the uploaded file available as table NEW_LABEL_TABLE_AINUGF in caslib CASUSER(chrobi).
NOTE: The table NEW_LABEL_TABLE_AINUGF has been created in caslib CASUSER(chrobi) from binary data uploaded to Cloud Analytic Services.
NOTE: no dataspec(s) provided - creating image classification model.
NOTE: Model weights attached successfully!<jupyter_text>Summarize the SAS Deep Learning Model ArchitecturePlease compare the SAS Deep Learning model architecture and parameter count in this summary to the model architecture and parameter count from the Keras model above. The model layers and model layer definitions may differ somewhat between the Keras and SAS Deep Learning models, but the total number of trained parameters in the two models should be identical.<jupyter_code>model1.print_summary()<jupyter_output><empty_output><jupyter_text>Load the MNIST Image Classification Test DataYou will need to provide a path to your environment's server path in order to load the MNIST test data images. This example uses the SERVER_IMAGE_ROOT server path variable that you specified earlier in this example.<jupyter_code># import SAS DLPy simple image processing functions
from dlpy.images import ImageTable

# load the MNIST image classification model test data images
my_images = ImageTable.load_files(s, path=server_image_root)<jupyter_output><empty_output><jupyter_text>Verify Appearance of Test Data ImagesAfter loading the MNIST model test data images into the SAS Viya model in DLPy, display a matrix of 8 random images arranged in 4 columns. Verify that the test images resemble various hand-written Arabic numerals.<jupyter_code>my_images.show(nimages=8,ncol=4, randomize=True)<jupyter_output><empty_output><jupyter_text>Score the Test Images with the SAS Deep Learning ModelNow use the SAS DLPy `predict()` function to score the test images using the Deep Learning classification model `model1`.<jupyter_code># score images
viya_score = model1.predict(my_images, buffer_size=batch_size, n_threads=1)
print(viya_score['ScoreInfo'])<jupyter_output>Descr         Value
0  Number of Observations Read         10000
1  Number of Observations Used         10000
2  Misclassification Error (%)          1.28
3                   Loss Error      0.037126<jupyter_text>Verify the ResultsCompare the classification results generated by the Keras model and the imported SAS Viya model.These results should be identical (or nearly so).  Note that the Keras model reports classification rate, while the SAS Deep Learning model reports misclassification rate. To compare classification rates, subtract the Deep Learning misclassification rate from 100. The code below calculates the classification rate in terms of the percent of images correctly classified.  The output shows that the original Keras image classification model and the imported SAS Deep Learning image classification model deliver identical classification performance.<jupyter_code>print('Viya classification rate = ' + "{:2.2f}".format(100.0-float(viya_score['ScoreInfo']['Value'][2])) + '%')
print('Keras classification rate = ' + "{:2.2f}".format(100.0*score[1]) + '%')<jupyter_output>Viya classification rate = 98.72%
Keras classification rate = 98.75%
{'fraction_non_alphanumeric': 0.0688757275605024, 'fraction_numerical': 0.021239660982334322, 'mean_word_length': 4.372188699945146, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '303470', 'license': 'Apache-2.0', 'n_tokens_mistral': 5566, 'n_tokens_neox': 5166, 'n_words': 2253, 'path': 'examples/keras_model_conversion/Import_Keras_LeNet.ipynb', 'repo_name': 'ChipRobie/python-dlpy', 'size': '19596'}