<jupyter_start><jupyter_text>Lab4. Training, Deploying and hosting custom model on SageMaker In this lab, you will run your training job with SageMaker. You will use your CNN architecture and script that you build at Lab3.    Set up SageMaker environment * **Plase note that the Tensorflow eager mode will be enabled by below code:**   tf.enable_eager_execution()<jupyter_code>from PIL import Image
import json
import os
import matplotlib.image as mpimg
import tensorflow as tf
from tensorflow.contrib.eager.python import tfe
from sagemaker.tensorflow import TensorFlow

tf.enable_eager_execution()
%%time
import sagemaker
from sagemaker import get_execution_role

role = get_execution_role()
print(role)
sess = sagemaker.Session()<jupyter_output>arn:aws:iam::308961792850:role/service-role/AmazonSageMaker-ExecutionRole-20180531T094549
CPU times: user 92 ms, sys: 478 µs, total: 92.5 ms
Wall time: 119 ms<jupyter_text>Upload data to S3 * **Make sure you change the bucket name None with your S3 bucket name**<jupyter_code>bucket = None #'leonkang-datalake-nv' # custom bucket name.
prefix = 'anpr'
assert bucket != None<jupyter_output><empty_output><jupyter_text>* Upload files to S3* It may take several minutes **(10~15 minutes for 10,000 images)**<jupyter_code>%%time

train_channel = prefix + '/tf/tf-train'
validation_channel = prefix + '/tf/tf-validation'
train_annotation_channel = prefix + '/tf/tf-train_annotation'
validation_annotation_channel = prefix + '/tf/tf-validation_annotation'

sess.upload_data(path='tf/tf-train', bucket=bucket, key_prefix=train_channel)
sess.upload_data(path='tf/tf-validation', bucket=bucket, key_prefix=validation_channel)
sess.upload_data(path='tf/tf-train_annotation', bucket=bucket, key_prefix=train_annotation_channel)
sess.upload_data(path='tf/tf-validation_annotation', bucket=bucket, key_prefix=validation_annotation_channel)

s3_train_data = 's3://{}/{}'.format(bucket, train_channel)
s3_validation_data = 's3://{}/{}'.format(bucket, validation_channel)
s3_train_annotation = 's3://{}/{}'.format(bucket, train_annotation_channel)
s3_validation_annotation = 's3://{}/{}'.format(bucket, validation_annotation_channel)<jupyter_output>CPU times: user 1min 53s, sys: 7.55 s, total: 2min 1s
Wall time: 14min 37s<jupyter_text>* Check upload results<jupyter_code>!aws s3 ls {bucket}/anpr/od/od-train/ |wc -l
!aws s3 ls {bucket}/anpr/od/od-train_annotation/ |wc -l
!aws s3 ls {bucket}/anpr/od/od-validation/ |wc -l
!aws s3 ls {bucket}/anpr/od/od-validation_annotation/ |wc -l
s3_output_location = 's3://{}/{}/tf-output'.format(bucket, prefix)<jupyter_output><empty_output><jupyter_text>Run training job on SageMaker Training data will be copied from S3 to Tensorflow container.  Refer to the below directory paths.  Test and validation data directories1) In this notebook- train images: './tf/tf-train'- train annotations: './tf/tf-train_annotations'- validation images: './tf/tf-validation'- validation annotations: './tf/tf-validation_annotations'2) S3- train images: 's3://leonkang-datalake-seoul/anpr/tf/tf-train'- train annotations: 's3://leonkang-datalake-seoul/anpr/tf/tf-validation'- validation images: 's3://leonkang-datalake-seoul/anpr/tf/tf-train_annotations'- validation annotations: 's3://leonkang-datalake-seoul/anpr/tf/tf-validation_annotations'**3) Training container (Tensorflow container)**- **train images: '/opt/ml/input/data/train/'**- **train annotations: '/opt/ml/input/data/train_annotation/'**- **validation images: '/opt/ml/input/data/validation/'**- **validation annotations: '/opt/ml/input/data/validation_annotation/'**<jupyter_code>train_data = sagemaker.session.s3_input(s3_train_data, distribution='FullyReplicated', 
                        content_type='image/png', s3_data_type='S3Prefix')
validation_data = sagemaker.session.s3_input(s3_validation_data, distribution='FullyReplicated', 
                             content_type='image/png', s3_data_type='S3Prefix')
train_annotation = sagemaker.session.s3_input(s3_train_annotation, distribution='FullyReplicated', 
                             content_type='application/json', s3_data_type='S3Prefix')
validation_annotation = sagemaker.session.s3_input(s3_validation_annotation, distribution='FullyReplicated', 
                             content_type='application/json', s3_data_type='S3Prefix')

data_channels = {'train': train_data, 'validation': validation_data, 
                 'train_annotation': train_annotation, 'validation_annotation':validation_annotation}<jupyter_output><empty_output><jupyter_text>* **This time, we will run 30 epochs to build more accurate model. (It may take less than 15 minutes)*** Note that we use p3.2xlarge instance, which will shorten our training time with GPU. * Refer to API specs of SDK site    * https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/README.rst    * https://sagemaker.readthedocs.io/en/latest/using_tf.html* Refer to the API document     * https://sagemaker.readthedocs.io/en/stable/sagemaker.tensorflow.html     (Note that checkpoint_path, training_steps, evaluation_steps, requirements_file args are not permitted when using Script Mode) * **Check the code of 'cnn_ocr.py' file**    * Notice the hyperparameters are parsed from argparse.ArgumentParser()    * Notice the model function is same as Lab3    * In the main function, input data are loaded from local directory 'opt/ml/data/{...channel...}'<jupyter_code>!cat cnn_ocr.py
model_dir = '/opt/ml/model'
train_instance_type='ml.p3.2xlarge'
hyperparameters = {'epochs': 30, 
                   'batch_size': 128,
                   'learning_rate': 0.001}
%%time
ocr_estimator = TensorFlow(entry_point='cnn_ocr.py',
                             role=role,
                             model_dir=model_dir,
                             hyperparameters=hyperparameters,
                             train_instance_count=2,
                             train_instance_type=train_instance_type,
                             distributions={'parameter_server': {'enabled': True}},
#                              distributions={
#                                   'mpi': {
#                                       'enabled': True,
#                                       'processes_per_host': 8,
#                                       'custom_mpi_options': '--NCCL_DEBUG INFO'
#                                   }
#                              }),
                             framework_version='1.12', 
                             py_version='py3')

ocr_estimator.fit(data_channels)<jupyter_output><empty_output><jupyter_text>* Check the model name that training job just created* Check the model file that training job just created<jupyter_code>print(ocr_estimator.output_path)
print(ocr_estimator.model_data)

!aws s3 ls {ocr_estimator.model_data}<jupyter_output>s3://sagemaker-us-east-1-308961792850/
s3://sagemaker-us-east-1-308961792850/sagemaker-tensorflow-scriptmode-2019-03-09-00-23-00-877/output/model.tar.gz
2019-03-09 00:34:47   75768194 model.tar.gz<jupyter_text>Test data * Prepare character list and test images<jupyter_code>import numpy as np
NUMS =['1', '2', '3', '4', '5', '6', '7', '8', '9', '0']
CHARS=['가', '나', '다', '라', '마', '거', '너', '더', '러', '머', '고', '노', '도', '로', '모', '구', '누', '두', '루', '무', 
       '버', '서', '어', '저', '처', '커', '터', '퍼', '보', '소', '오', '조', '초', '코', '토', '포', '부', '수', '우', '주', '추', '쿠', '투', '푸', '후', 
       '그', '느', '드', '르', '므', '브', '스', '으', '즈', '츠', '크', '트', '프', '흐',
       '바', '사', '아', '자', '차', '카', '타', '파', '하', '허', '호']
SPACE=[' ']
JOIN =NUMS + CHARS + SPACE
n_chr = len(JOIN)
test_imgs = os.listdir('tf/tf-validation')<jupyter_output><empty_output><jupyter_text>Test the trainned model in local * Copy trainned model from S3 to ./model folder<jupyter_code>#!rm -Rf model
!aws s3 cp {ocr_estimator.model_data} ./model/model.tar.gz --region us-east-1<jupyter_output><jupyter_text>* Extract the model file and see the components<jupyter_code>!tar -xvzf ./model/model.tar.gz -C ./model<jupyter_output>1552091667/
1552091667/variables/
1552091667/variables/variables.index
1552091667/variables/variables.data-00000-of-00001
1552091667/variables/checkpoint
1552091667/assets/
1552091667/assets/saved_model.json
1552091667/saved_model.pb
weights.ckpt.data-00000-of-00001
weights.ckpt.index
checkpoint
weights.ckpt.index
weights.ckpt.data-00000-of-00001
checkpoint
1552091672/
1552091672/assets/
1552091672/assets/saved_model.json
1552091672/saved_model.pb
1552091672/variables/
1552091672/variables/variables.data-00000-of-00001
1552091672/variables/variables.index
1552091672/variables/checkpoint<jupyter_text>* Define model architecture locally (same as Lab3)<jupyter_code>import matplotlib.pyplot as plt
def model(input_shape):
    # Define the input placeholder as a tensor with shape input_shape. Think of this as your input image!
    X_input = tf.keras.Input(input_shape)

    # Zero-Padding: pads the border of X_input with zeroes
    X = tf.keras.layers.ZeroPadding2D((2, 2))(X_input)

    # CONV0 -> BN -> RELU -> MAXPOOL applied to X
    X = tf.keras.layers.Conv2D(48, (5, 5), strides = (1, 1), name = 'conv0')(X)
    X = tf.keras.layers.BatchNormalization(axis = 3, name = 'bn0')(X)
    X = tf.keras.layers.Activation('relu')(X)
    X = tf.keras.layers.MaxPooling2D((2, 2), strides = (2, 2), name='max_pool0')(X)

    # CONV1 -> BN -> RELU -> MAXPOOL applied to X
    X = tf.keras.layers.Conv2D(64, (5, 5), strides = (1, 1), name = 'conv1')(X)
    X = tf.keras.layers.BatchNormalization(axis = 3, name = 'bn1')(X)
    X = tf.keras.layers.Activation('relu')(X)
    X = tf.keras.layers.MaxPooling2D((1, 2), strides = (2, 2), name='max_pool1')(X)

    # CONV2 -> BN -> RELU -> MAXPOOL applied to X
    X = tf.keras.layers.Conv2D(64, (5, 5), strides = (1, 1), name = 'conv2')(X)
    X = tf.keras.layers.BatchNormalization(axis = 3, name = 'bn2')(X)
    X = tf.keras.layers.Activation('relu')(X)
    X = tf.keras.layers.MaxPooling2D((2, 2), strides = (2, 2), name='max_pool2')(X)

    # FLATTEN X (means convert it to a vector) + FULLYCONNECTED
    X = tf.keras.layers.Flatten()(X)
    #----------------------------------------------------------
    d1 = tf.keras.layers.Dense(81, activation='softmax', name='d1')(X)
    d2 = tf.keras.layers.Dense(81, activation='softmax', name='d2')(X)
    d3 = tf.keras.layers.Dense(81, activation='softmax', name='d3')(X)
    d4 = tf.keras.layers.Dense(81, activation='softmax', name='d4')(X)
    d5 = tf.keras.layers.Dense(81, activation='softmax', name='d5')(X)
    d6 = tf.keras.layers.Dense(81, activation='softmax', name='d6')(X)
    d7 = tf.keras.layers.Dense(81, activation='softmax', name='d7')(X)
    
    model = tf.keras.Model(inputs = X_input, 
                  outputs = [d1,d2,d3,d4,d5,d6,d7],
                  name='CNN_ANPR')

    return model<jupyter_output><empty_output><jupyter_text>* Read and retore model from downloaded checkpoint<jupyter_code>tf.keras.backend.clear_session()
device = '/cpu:0' 

with tf.device(device):    
    local_model = model((128,64,1))
    saver = tfe.Saver(local_model.variables)
    saver.restore('model/weights.ckpt')
for i in range(10):
    val_img = Image.open('tf/tf-validation/' + test_imgs[i])
    val_img_np = np.array(val_img.resize((128,64)))
    prediction = local_model.predict(val_img_np.reshape([1,128,64,1])/255.)
    plt.imshow(val_img)
    plt.show()
    digits = [JOIN[np.argmax(prediction[0][0])], 
              JOIN[np.argmax(prediction[1][0])], 
              JOIN[np.argmax(prediction[2][0])], 
              JOIN[np.argmax(prediction[3][0])], 
              JOIN[np.argmax(prediction[4][0])], 
              JOIN[np.argmax(prediction[5][0])], 
              JOIN[np.argmax(prediction[6][0])]]
    print(digits)<jupyter_output><empty_output><jupyter_text>* You may utilize above code to build your own inference environment* You can find that the accuracy of our model is increased slightly* We may further refine our model in the following ways:    * Adding more training images and more epochs    * Adding real world license plate images    * Changing an architecture to another structure (ex: CNN + LSTM)    * Combining Object detection and this OCR model into one CNN architecture (Usually, end-to-end approach needs more training data)  Deploy the model and inference Test * Now, let's deploy our model into SageMaker's inference hosting environment<jupyter_code>predictor = ocr_estimator.deploy(initial_instance_count=1,
                                 instance_type='ml.m4.xlarge',
                                 accelerator_type='ml.eia1.medium')<jupyter_output>INFO:sagemaker:Creating model with name: sagemaker-tensorflow-scriptmode-2019-03-09-00-23-00-877
INFO:sagemaker:Creating endpoint with name sagemaker-tensorflow-scriptmode-2019-03-09-00-23-00-877<jupyter_text>* Check the result of deploy<jupyter_code>predictor
predictor.endpoint<jupyter_output><empty_output><jupyter_text>* Check the prediction of 10 sample pairs of images<jupyter_code>for i in range(10):
    val_img = Image.open('tf/tf-validation/' + test_imgs[i])
    val_img_np = np.array(val_img.resize((128,64)))
    input = {
      'instances': val_img_np.reshape([1,128,64,1])/255.
    }
    prediction = predictor.predict(input)
    plt.imshow(val_img)
    plt.show()
    print(i)
    digits = [JOIN[np.argmax(prediction['predictions'][0]['d1'])], 
              JOIN[np.argmax(prediction['predictions'][0]['d2'])], 
              JOIN[np.argmax(prediction['predictions'][0]['d3'])], 
              JOIN[np.argmax(prediction['predictions'][0]['d4'])], 
              JOIN[np.argmax(prediction['predictions'][0]['d5'])],  
              JOIN[np.argmax(prediction['predictions'][0]['d6'])], 
              JOIN[np.argmax(prediction['predictions'][0]['d7'])]]
    print(digits)<jupyter_output><empty_output><jupyter_text>Endpoint API test * You may invoke SageMaker endpoint from business application as below:<jupyter_code>import boto3
from sagemaker.predictor import json_serializer, json_deserializer, RealTimePredictor
from sagemaker.content_types import CONTENT_TYPE_CSV, CONTENT_TYPE_JSON
runtime = boto3.Session().client('runtime.sagemaker')
tf_endpoint = predictor.endpoint
api_predictor = RealTimePredictor(endpoint=tf_endpoint, sagemaker_session=sess, serializer=json_serializer,
                                content_type=CONTENT_TYPE_JSON, accept=CONTENT_TYPE_CSV)
api_response = json.loads(api_predictor.predict(input))
print(JOIN[np.argmax(api_response['predictions'][0]['d1'])] +
      JOIN[np.argmax(api_response['predictions'][0]['d2'])] +
      JOIN[np.argmax(api_response['predictions'][0]['d3'])] +
      JOIN[np.argmax(api_response['predictions'][0]['d4'])] +
      JOIN[np.argmax(api_response['predictions'][0]['d5'])] +
      JOIN[np.argmax(api_response['predictions'][0]['d6'])] +
      JOIN[np.argmax(api_response['predictions'][0]['d7'])] )<jupyter_output>94터6322<jupyter_text>Delete Endpoint<jupyter_code>sagemaker.Session().delete_endpoint(predictor.endpoint)<jupyter_output>INFO:sagemaker:Deleting endpoint with name: sagemaker-tensorflow-scriptmode-2019-03-09-00-23-00-877
{'fraction_non_alphanumeric': 0.1472627616720597, 'fraction_numerical': 0.042527900680182265, 'mean_word_length': 3.916883116883117, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 70, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '245602', 'license': 'Net-SNMP', 'n_tokens_mistral': 5565, 'n_tokens_neox': 5245, 'n_words': 1036, 'path': 'Lab4-train-deploy-host-on-SM.ipynb', 'repo_name': 'jihys/lab-custom-model-anpr', 'size': '15143'}