<jupyter_start><jupyter_text>Brainwave Service UploadThis Notebook is intended to simplify the training / upload process by splitting the two steps into two separate notebooks. In particular, this Notebook is for uploading a previously-trained model to the cloud, and doesn't contain any training code. It does have some sanity check code to ensure you're loading in the right model, before actually uploading it. Even though these first few cells are repeated in the training Notebooks, it is necessary here since we still must set up the environment to load the model into memory.<jupyter_code>import os,sys
os.environ['KERAS_BACKEND'] = 'tensorflow'
os.environ['CUDA_VISIBLE_DEVICES'] = ''
import tensorflow as tf
import numpy as np
from keras import backend as K
import tables
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
%load_ext autoreload
%autoreload 2
# Directories for models
custom_weights_dir_q = os.path.expanduser("../weights-quantized-224x224-fixval-best-final/")
saved_model_dir = os.path.expanduser("../machinelearningnotebooks/models/")<jupyter_output><empty_output><jupyter_text>Prepare DataLoad the files we are going to use for training and testing. The public Top dataset consists of image formatted data, but our data has been preprocessed into a raw form. You will need to edit the paths as necessary.<jupyter_code>from utils import normalize_and_rgb, image_with_label, count_events
import glob
# for 224x224:
datadir = "../../converted/rotation_224_v1/"
data_size = 224 #image width/height
n_test_file = 1
test_files = glob.glob(os.path.join(datadir, 'test_file_*'))
n_test_events = count_events(test_files)
print("n_test_events =", n_test_events)
def test_heatmap(train_files, size=224):
    import matplotlib as mpl
    import matplotlib.pyplot as plt
    import numpy as np
    
    a, b = image_with_label(train_files[0],0,100)
    a = a[0:100:60] # pick one top image [0] and one QCD image [59]
    b = b[0:100:60] 
    new_a = np.swapaxes(a[:,:,:,0],0,2)
    new_a = np.swapaxes(new_a,0,1)
    # pick top image
    c = np.dot(new_a,b[:,0])/a.shape[0]
    # pick QCD image
    d = np.dot(new_a,b[:,1])/a.shape[0]

    width = 12
    height = 12
    fontsize = 30

    plt.style.use('sonic.mplstyle')
    plt.figure(figsize=(width,height))
    ax = plt.subplot() 
    for label in (ax.get_xticklabels() + ax.get_yticklabels()): label.set_fontsize(fontsize)
    plt.imshow(c, norm=mpl.colors.LogNorm(), origin='lower', interpolation='nearest',label='top')
    cbar = plt.colorbar(shrink=0.675)
    cbar.ax.tick_params(labelsize=fontsize)
    cbar.set_label(r'Normalized $p_{\mathrm{T}}$ [AU]', fontsize=fontsize)
    plt.xlabel(r'$i\eta$', fontsize=fontsize)
    plt.ylabel(r'$i\phi$', fontsize=fontsize)
    plt.tight_layout()
    plt.savefig('top_%sx%s.pdf'%(size,size))

    plt.style.use('sonic.mplstyle')
    plt.figure(figsize=(width,height))
    ax = plt.subplot() 
    for label in (ax.get_xticklabels() + ax.get_yticklabels()): label.set_fontsize(fontsize)
    plt.imshow(d, norm=mpl.colors.LogNorm(), origin='lower', interpolation='nearest',label='QCD')
    cbar = plt.colorbar(shrink=0.675)
    cbar.ax.tick_params(labelsize=fontsize)
    cbar.set_label(r'Normalized $p_{\mathrm{T}}$ [AU]', fontsize=fontsize)
    plt.xlabel(r'$i\eta$', fontsize=fontsize)
    plt.ylabel(r'$i\phi$', fontsize=fontsize)
    plt.tight_layout()
    plt.savefig('QCD_%sx%s.pdf'%(size,size))
%matplotlib inline
test_heatmap(test_files,224)<jupyter_output>/home/jduarte/miniconda/envs/myamlenv/lib/python3.6/site-packages/matplotlib/figure.py:2366: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.
  warnings.warn("This figure includes Axes that are not compatible "<jupyter_text>Load The Model<jupyter_code>from utils import construct_model
tf.reset_default_graph()
sess = tf.Session(graph=tf.get_default_graph())

with sess.as_default():
    print("Loading quantized model")
    in_images, image_tensors, features, preds, quantized_featurizer, classifier = construct_model(quantized=True, 
                                                                                                  saved_model_dir=saved_model_dir, 
                                                                                                  starting_weights_directory=custom_weights_dir_q, 
                                                                                                  is_training=False, 
                                                                                                  size=data_size)<jupyter_output>Loading quantized model
INFO:tensorflow:Restoring parameters from ../weights-quantized-224x224-fixval-best-final/resnet50_bw_best
loading classifier weights from ../weights-quantized-224x224-fixval-best-final//class_weights_best.h5<jupyter_text>Service DefinitionLike in the QuickStart notebook our service definition pipeline consists of three stages.<jupyter_code>from azureml.contrib.brainwave.pipeline import ModelDefinition, TensorflowStage, BrainWaveStage

model_def_path = os.path.join(saved_model_dir, 'model_def.zip')

model_def = ModelDefinition()
model_def.pipeline.append(BrainWaveStage(sess, quantized_featurizer))
model_def.pipeline.append(TensorflowStage(sess, features, preds)) # comment this out to get the features
model_def.save(model_def_path)
print(model_def_path)<jupyter_output>INFO:tensorflow:Restoring parameters from ../weights-quantized-224x224-fixval-best-final/resnet50_bw_best
INFO:tensorflow:Froze 4 variables.
Converted 4 variables to const ops.
../machinelearningnotebooks/models/model_def.zip<jupyter_text>DeployGo to our GitHub repo "docs" folder to learn how to create a Model Management Account and find the required information below.<jupyter_code>import os

subscription_id = os.environ.get("SUBSCRIPTION_ID", "80defacd-509e-410c-9812-6e52ed6a0016")
resource_group = os.environ.get("RESOURCE_GROUP", "CMS_FPGA_Resources")
workspace_name = os.environ.get("WORKSPACE_NAME", "Fermilab")
from azureml.core import Workspace

try:
   ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name = workspace_name)
   ws.write_config()
   print('Workspace configuration succeeded. You are all set!')
except:
   print('Workspace not found. Run the cells below.')
from azureml.core import Workspace

ws = Workspace.from_config()
print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\n')<jupyter_output>Found the config file in: /home/jduarte/MachineLearningNotebooks/project-brainwave/aml_config/config.json
Fermilab
CMS_FPGA_Resources
eastus2
80defacd-509e-410c-9812-6e52ed6a0016<jupyter_text>The first time the code below runs it will create a new service running your model. If you want to change the model you can make changes above in this notebook and save a new service definition. Then this code will update the running service in place to run the new model.<jupyter_code>from azureml.core.model import Model
from azureml.core.image import Image
from azureml.core.webservice import Webservice
from azureml.contrib.brainwave import BrainwaveWebservice, BrainwaveImage
from azureml.exceptions import WebserviceException

model_name = "top-transfer-resnet50-model"
image_name = "top-transfer-resnet50-image"
service_name = "modelbuild-service-10"

registered_model = Model.register(ws, model_def_path, model_name)

image_config = BrainwaveImage.image_configuration()
deployment_config = BrainwaveWebservice.deploy_configuration()
    
try:
    service = Webservice(ws, service_name)
    service.delete()
    service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)
    service.wait_for_deployment(True)
except WebserviceException:
    service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)
    service.wait_for_deployment(True)<jupyter_output><empty_output><jupyter_text>The service is now running in Azure and ready to serve requests. We can check the address and port.<jupyter_code>from azureml.core.webservice import Webservice
service_name = "modelbuild-service-10"
service = Webservice(ws, service_name)
print(service.ip_address + ':' + str(service.port))<jupyter_output><empty_output><jupyter_text>ClientThere is a simple test client at amlrealtimeai.PredictionClient which can be used for testing. We'll use this client to score an image with our new service.<jupyter_code>from azureml.contrib.brainwave.client import PredictionClient
client = PredictionClient(service.ip_address, service.port)<jupyter_output><empty_output><jupyter_text>RequestLet's see how our service does on a few images. It may get a few wrong.<jupyter_code>from sklearn.metrics import roc_auc_score, accuracy_score, roc_curve
from tqdm import tqdm
from utils import chunks

chunk_size = 1  # Brainwave only processes one request at a time
n_test_events = count_events(test_files)
chunk_num = int(n_test_events/chunk_size)+1

y_true = np.zeros((n_test_events,2))
y_pred = np.zeros((n_test_events,2))

i = 0
for img_chunk, label_chunk, real_chunk_size in tqdm(chunks(test_files, chunk_size, max_q_size=1, shuffle=False), total=chunk_num):
    results = client.score_numpy_array(img_chunk)
    y_pred[i,:] = results
    y_true[i,:] = label_chunk
    print(results, y_true)
    i+=1
    if i>=100: break
# Call the save results utility.
from utils import save_results

accuracy = accuracy_score(y_true[:,0], y_pred[:,0]>0.5)
auc = roc_auc_score(y_true, y_pred)

print("Accuracy:", accuracy, "AUC:", auc)<jupyter_output><empty_output><jupyter_text>CleanupRun the cell below to delete your service.<jupyter_code>service.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0970714724554577, 'fraction_numerical': 0.01679295515052222, 'mean_word_length': 4.626152073732719, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '658157', 'license': 'MIT', 'n_tokens_mistral': 3221, 'n_tokens_neox': 3070, 'n_words': 749, 'path': 'project-brainwave/project-brainwave-custom-weights-service-upload-demo.ipynb', 'repo_name': 'fastmachinelearning/MachineLearningNotebooks', 'size': '9766'}