<jupyter_start><jupyter_text>5 Train an image classification model with Azure Machine Learning![](https://github.com/ashleymcnamara/Developer-Advocate-Bit/blob/master/BIT_ML.png?raw=true)Photo credit to the talented <NAME> Sometimes you need more computational power then locally available to train a model. Let's Train  a  Computer Vision model to the Cloud using the Azure Machine Learning Service to make this simple we will revisit the MNIST dataset from the third NotebookBased on the [Azure ML Documentation](https://docs.microsoft.com/en-us/azure/machine-learning/service/tutorial-train-models-with-aml?WT.mc_id=cvworkshop-github-abornst)In this notebook you will learn how to:> * Set up your development environment> * Access and examine the data> * Train a simple logistic regression model locally using the popular scikit-learn machine learning library > * Train multiple models on a remote cluster> * Review training results, find and register the best modelYou'll learn how to select a model and deploy it in part 6 of the workshop  Import packagesImport Python packages you need in this session. Also display the Azure Machine Learning SDK version.<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

import azureml
from azureml.core import Workspace, Run

# check core SDK version number
print("Azure ML SDK Version: ", azureml.core.VERSION)<jupyter_output>Azure ML SDK Version:  1.0.2<jupyter_text>Login and Authenticate with Azure Generate credientials file becareful with this anyone who has acess to this file has acess to your account.<jupyter_code>!python3 -m azure.cli login
!python3 -m azure.cli ad sp create-for-rbac --sdk-auth > mycredentials.json
!export AZURE_AUTH_LOCATION='mycredentials.json'
import os, json
with open('mycredentials.json') as data_file:    
    azure_credentials = json.load(data_file)

# delete credentials file 
os.remove("mycredentials.json")<jupyter_output><empty_output><jupyter_text>What is an Azure ML Workspace and why do I need one?An AML Workspace is an Azure resource that organizes and coordinates the actions of many other Azure resources to assist in executing and sharing machine learning workflows.  In particular, an AML Workspace coordinates storage, databases, and compute resources providing added functionality for machine learning experimentation, operationalization, and the monitoring of operationalized models.<jupyter_code>CVWorkshop_RG = 'cvdemo_rg'
CVWorkshop_LOC = 'eastus2'
CVWorkshop_WS = 'cv_ws'<jupyter_output><empty_output><jupyter_text>Set up your Azure Machine Learning workspace Option 1: You have workspace alreadyIf you ran the Azure Machine Learning [quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-get-started) in Azure Notebooks, you already have a configured workspace!  You can go to your Azure Machine Learning Getting Started library, view *config.json* file, and copy-paste the values for subscription ID, resource group and workspace name below.If you have a workspace created another way, [these instructions](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-environmentcreate-workspace-configuration-file) describe how to get your subscription and workspace information.If this cell succeeds, you're done configuring this library!  Otherwise continue to follow the instructions in the rest of the notebook.<jupyter_code>import os

subscription_id = os.environ.get("SUBSCRIPTION_ID", azure_credentials['subscriptionId'])
resource_group = os.environ.get("RESOURCE_GROUP", CVWorkshop_RG)
workspace_name = os.environ.get("WORKSPACE_NAME", CVWorkshop_WS)

from azureml.core import Workspace

try:
    ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name = workspace_name)
    ws.write_config()
    print('Workspace configuration succeeded. You are all set!')
except:
    print('Workspace not found. Run the cells below.')<jupyter_output><empty_output><jupyter_text>Option 2: You don't have workspace yet RequirementsInside your Azure subscription, you will need access to a _resource group_, which organizes Azure resources and provides a default region for the resources in a group.  You can see what resource groups to which you have access, or create a new one in the [Azure portal](https://portal.azure.com).  If you don't have a resource group, the create workspace command will create one for you using the name you provide.To create or access an Azure ML Workspace, you will need to import the AML library and the following information:* A name for your workspace* Your subscription id* The resource group name Supported Azure RegionsSpecify a region where your workspace will be located from the list of [Azure Machine Learning regions](https://azure.microsoft.com/en-us/global-infrastructure/services/)<jupyter_code># import the Workspace class and check the azureml SDK version
from azureml.core import Workspace

ws = Workspace.create(name = CVWorkshop_WS,
                      subscription_id = azure_credentials['subscriptionId'],
                      resource_group = CVWorkshop_RG, 
                      location = CVWorkshop_LOC,
                      create_resource_group = True,
                      exist_ok = True)
ws.get_details()
ws.write_config()<jupyter_output><empty_output><jupyter_text>Connect to workspaceCreate a workspace object from the existing workspace. `Workspace.from_config()` reads the file **config.json** and loads the details into an object named `ws`.<jupyter_code># load workspace configuration from the config.json file in the current folder.
ws = Workspace.from_config()
print(ws.name, ws.location, ws.resource_group, ws.location, sep = '\t')<jupyter_output>Found the config file in: /data/aml_config/config.json
cv_ws	eastus2	cvdemo_rg	eastus2<jupyter_text>Create experimentCreate an experiment to track the runs in your workspace. A workspace can have muliple experiments.<jupyter_code>experiment_name = 'sklearn-mnist'

from azureml.core import Experiment
exp = Experiment(workspace=ws, name=experiment_name)<jupyter_output><empty_output><jupyter_text>Create or Attach existing AmlComputeYou will need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecturecompute-target) for training your model. In this tutorial, you create `AmlCompute` as your training compute resource.**Creation of AmlCompute takes approximately 5 minutes.** If the AmlCompute with that name is already in your workspace this code will skip the creation process.<jupyter_code>from azureml.core.compute import AmlCompute
from azureml.core.compute import ComputeTarget
import os

# choose a name for your cluster
compute_name = os.environ.get("AML_COMPUTE_CLUSTER_NAME", "cpucluster")
compute_min_nodes = os.environ.get("AML_COMPUTE_CLUSTER_MIN_NODES", 0)
compute_max_nodes = os.environ.get("AML_COMPUTE_CLUSTER_MAX_NODES", 4)

# This example uses CPU VM. For using GPU VM, set SKU to STANDARD_NC6
vm_size = os.environ.get("AML_COMPUTE_CLUSTER_SKU", "STANDARD_D2_V2")


if compute_name in ws.compute_targets:
    compute_target = ws.compute_targets[compute_name]
    if compute_target and type(compute_target) is AmlCompute:
        print('found compute target. just use it. ' + compute_name)
else:
    print('creating a new compute target...')
    provisioning_config = AmlCompute.provisioning_configuration(vm_size = vm_size,
                                                                min_nodes = compute_min_nodes, 
                                                                max_nodes = compute_max_nodes)

    # create the cluster
    compute_target = ComputeTarget.create(ws, compute_name, provisioning_config)
    
    # can poll for a minimum number of nodes and for a specific timeout. 
    # if no min node count is provided it will use the scale settings for the cluster
    compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)
    
     # For a more detailed view of current AmlCompute status, use the 'status' property    
    print(compute_target.status.serialize())<jupyter_output>creating a new compute target...
Creating
Succeeded
AmlCompute wait for completion finished
Minimum number of nodes requested have been provisioned
{'vmSize': 'STANDARD_D2_V2', 'modifiedTime': '2018-12-08T08:23:03.182849+00:00', 'nodeStateCounts': {'preemptedNodeCount': 0, 'idleNodeCount': 0, 'runningNodeCount': 0, 'preparingNodeCount': 0, 'unusableNodeCount': 0, 'leavingNodeCount': 0}, 'currentNodeCount': 0, 'targetNodeCount': 0, 'provisioningState': 'Succeeded', 'provisioningStateTransitionTime': None, 'allocationState': 'Steady', 'allocationStateTransitionTime': '2018-12-08T08:22:58.550000+00:00', 'scaleSettings': {'nodeIdleTimeBeforeScaleDown': 'PT120S', 'maxNodeCount': 4, 'minNodeCount': 0}, 'vmPriority': 'Dedicated', 'creationTime': '2018-12-08T08:21:10.796522+00:00', 'errors': None}<jupyter_text>You now have the necessary packages and compute resources to train a model in the cloud.  Explore dataBefore you train a model, you need to understand the data that you are using to train it.  You also need to copy the data into the cloud so it can be accessed by your cloud training environment.  In this section you learn how to:* Download the MNIST dataset* Display some sample images* Upload data to the cloud Download the MNIST datasetDownload the MNIST dataset and save the files into a `data` directory locally.  Images and labels for both training and testing are downloaded.<jupyter_code>import os
import urllib.request

os.makedirs('./data', exist_ok = True)

urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz', filename='./data/train-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz', filename='./data/train-labels.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz', filename='./data/test-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz', filename='./data/test-labels.gz')<jupyter_output><empty_output><jupyter_text>Display some sample imagesLoad the compressed files into `numpy` arrays. Then use `matplotlib` to plot 30 random images from the dataset with their labels above them. Note this step requires a `load_data` function that's included in an `util.py` file. This file is included in the sample folder. Please make sure it is placed in the same folder as this notebook. The `load_data` function simply parses the compresse files into numpy arrays.<jupyter_code>from utils import load_data

# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the model converge faster.
X_train = load_data('./data/train-images.gz', False) / 255.0
y_train = load_data('./data/train-labels.gz', True).reshape(-1)

X_test = load_data('./data/test-images.gz', False) / 255.0
y_test = load_data('./data/test-labels.gz', True).reshape(-1)

# now let's show some randomly chosen images from the traininng set.
count = 0
sample_size = 30
plt.figure(figsize = (16, 6))
for i in np.random.permutation(X_train.shape[0])[:sample_size]:
    count = count + 1
    plt.subplot(1, sample_size, count)
    plt.axhline('')
    plt.axvline('')
    plt.text(x=10, y=-10, s=y_train[i], fontsize=18)
    plt.imshow(X_train[i].reshape(28, 28), cmap=plt.cm.Greys)
plt.show()<jupyter_output><empty_output><jupyter_text>Now you have an idea of what these images look like and the expected prediction outcome. Upload data to the cloudNow make the data accessible remotely by uploading that data from your local machine into Azure so it can be accessed for remote training. The datastore is a convenient construct associated with your workspace for you to upload/download data, and interact with it from your remote compute targets. It is backed by Azure blob storage account.The MNIST files are uploaded into a directory named `mnist` at the root of the datastore.<jupyter_code>ds = ws.get_default_datastore()
print(ds.datastore_type, ds.account_name, ds.container_name)

ds.upload(src_dir='./data', target_path='mnist', overwrite=True, show_progress=True)<jupyter_output>AzureBlob cvwsstoragesiuuxjnq azureml-blobstore-0a19701f-cf5f-4472-88cc-093996fea2a5<jupyter_text>You now have everything you need to start training a model.   Train on a remote Azure ML clusterFor this task, submit the job to the remote training cluster you set up earlier.  To submit a job you:* Create a directory* Create a training script* Create an estimator object* Submit the job  Create a directoryCreate a directory to deliver the necessary code from your computer to the remote resource.<jupyter_code>import os
script_folder = './sklearn-mnist'
os.makedirs(script_folder, exist_ok=True)<jupyter_output><empty_output><jupyter_text>Create a training scriptTo submit the job to the cluster, first create a training script. Run the following code to create the training script called `train.py` in the directory you just created. This training adds a regularization rate to the training algorithm, so produces a slightly different model than the local version.<jupyter_code>%%writefile $script_folder/train.py

import argparse
import os
import numpy as np

from sklearn.linear_model import LogisticRegression
from sklearn.externals import joblib

from azureml.core import Run
from utils import load_data

# let user feed in 2 parameters, the location of the data files (from datastore), and the regularization rate of the logistic regression model
parser = argparse.ArgumentParser()
parser.add_argument('--data-folder', type=str, dest='data_folder', help='data folder mounting point')
parser.add_argument('--regularization', type=float, dest='reg', default=0.01, help='regularization rate')
args = parser.parse_args()

data_folder = os.path.join(args.data_folder, 'mnist')
print('Data folder:', data_folder)

# load train and test set into numpy arrays
# note we scale the pixel intensity values to 0-1 (by dividing it with 255.0) so the model can converge faster.
X_train = load_data(os.path.join(data_folder, 'train-images.gz'), False) / 255.0
X_test = load_data(os.path.join(data_folder, 'test-images.gz'), False) / 255.0
y_train = load_data(os.path.join(data_folder, 'train-labels.gz'), True).reshape(-1)
y_test = load_data(os.path.join(data_folder, 'test-labels.gz'), True).reshape(-1)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep = '\n')

# get hold of the current run
run = Run.get_context()

print('Train a logistic regression model with regularizaion rate of', args.reg)
clf = LogisticRegression(C=1.0/args.reg, random_state=42)
clf.fit(X_train, y_train)

print('Predict the test set')
y_hat = clf.predict(X_test)

# calculate accuracy on the prediction
acc = np.average(y_hat == y_test)
print('Accuracy is', acc)

run.log('regularization rate', np.float(args.reg))
run.log('accuracy', np.float(acc))

os.makedirs('outputs', exist_ok=True)
# note file saved in the outputs folder is automatically uploaded into experiment record
joblib.dump(value=clf, filename='outputs/sklearn_mnist_model.pkl')<jupyter_output>Writing ./sklearn-mnist/train.py<jupyter_text>Notice how the script gets data and saves models:+ The training script reads an argument to find the directory containing the data.  When you submit the job later, you point to the datastore for this argument:`parser.add_argument('--data-folder', type=str, dest='data_folder', help='data directory mounting point')` + The training script saves your model into a directory named outputs. `joblib.dump(value=clf, filename='outputs/sklearn_mnist_model.pkl')`Anything written in this directory is automatically uploaded into your workspace. You'll access your model from this directory later in the tutorial. The file `utils.py` is referenced from the training script to load the dataset correctly.  Copy this script into the script folder so that it can be accessed along with the training script on the remote resource.<jupyter_code>import shutil
shutil.copy('utils.py', script_folder)<jupyter_output><empty_output><jupyter_text>Create an estimatorAn estimator object is used to submit the run.  Create your estimator by running the following code to define:* The name of the estimator object, `est`* The directory that contains your scripts. All the files in this directory are uploaded into the cluster nodes for execution. * The compute target.  In this case you will use the AmlCompute you created* The training script name, train.py* Parameters required from the training script * Python packages needed for trainingIn this tutorial, this target is AmlCompute. All files in the script folder are uploaded into the cluster nodes for execution. The data_folder is set to use the datastore (`ds.as_mount()`).<jupyter_code>from azureml.train.estimator import Estimator

script_params = {
    '--data-folder': ds.as_mount(),
    '--regularization': 0.8
}

est = Estimator(source_directory=script_folder,
                script_params=script_params,
                compute_target=compute_target,
                entry_script='train.py',
                conda_packages=['scikit-learn','matplotlib'])<jupyter_output><empty_output><jupyter_text>Submit the job to the clusterRun the experiment by submitting the estimator object.<jupyter_code>run = exp.submit(config=est)
run<jupyter_output><empty_output><jupyter_text>Since the call is asynchronous, it returns a **Preparing** or **Running** state as soon as the job is started. Monitor a remote runIn total, the first run takes **approximately 10 minutes**. But for subsequent runs, as long as the script dependencies don't change, the same image is reused and hence the container start up time is much faster.Here is what's happening while you wait:- **Image creation**: A Docker image is created matching the Python environment specified by the estimator. The image is uploaded to the workspace. Image creation and uploading takes **about 5 minutes**.   This stage happens once for each Python environment since the container is cached for subsequent runs.  During image creation, logs are streamed to the run history. You can monitor the image creation progress using these logs.- **Scaling**: If the remote cluster requires more nodes to execute the run than currently available, additional nodes are added automatically. Scaling typically takes **about 5 minutes.**- **Running**: In this stage, the necessary scripts and files are sent to the compute target, then data stores are mounted/copied, then the entry_script is run. While the job is running, stdout and the ./logs directory are streamed to the run history. You can monitor the run's progress using these logs.- **Post-Processing**: The ./outputs directory of the run is copied over to the run history in your workspace so you can access these results.You can check the progress of a running job in multiple ways. This tutorial uses a Jupyter widget as well as a `wait_for_completion` method.  Jupyter widgetWatch the progress of the run with a Jupyter widget.  Like the run submission, the widget is asynchronous and provides live updates every 10-15 seconds until the job completes.<jupyter_code>from azureml.widgets import RunDetails
RunDetails(run).show()<jupyter_output><empty_output><jupyter_text>Get log results upon completionModel training and monitoring happen in the background. Wait until the model has completed training before running more code. Use `wait_for_completion` to show when the model training is complete.<jupyter_code>run.wait_for_completion(show_output=True) # specify True for a verbose log<jupyter_output>RunId: sklearn-mnist_1544258427260

Streaming azureml-logs/20_image_build_log.txt
=============================================

2018/12/08 08:40:35 Using acb_vol_501d07a8-bbe7-4ece-9965-d4856442d9d5 as the home volume
2018/12/08 08:40:35 Creating Docker network: acb_default_network, driver: 'bridge'
2018/12/08 08:40:35 Successfully set up Docker network: acb_default_network
2018/12/08 08:40:35 Setting up Docker configuration...
2018/12/08 08:40:36 Successfully set up Docker configuration
2018/12/08 08:40:36 Logging in to registry: cvwsacriceacvut.azurecr.io
2018/12/08 08:40:37 Successfully logged in
2018/12/08 08:40:37 Executing step ID: acb_step_0. Working directory: '', Network: 'acb_default_network'
2018/12/08 08:40:37 Obtaining source code and scanning for dependencies...
2018/12/08 08:40:38 Successfully obtained source code and scanned for dependencies
Sending build context to Docker daemon  139.8kB

Step 1/13 : FROM mcr.microsoft.com/azureml/base:0.2.0
0.2.0: Pulling from azurem[...]<jupyter_text>Display run resultsYou now have a model trained on a remote cluster.  Retrieve the accuracy of the model:<jupyter_code>print(run.get_metrics())<jupyter_output>{'accuracy': 0.9204, 'regularization rate': 0.8}<jupyter_text>In the next tutorial you will explore this model in more detail. Register modelThe last step in the training script wrote the file `outputs/sklearn_mnist_model.pkl` in a directory named `outputs` in the VM of the cluster where the job is executed. `outputs` is a special directory in that all content in this  directory is automatically uploaded to your workspace.  This content appears in the run record in the experiment under your workspace. Hence, the model file is now also available in your workspace.You can see files associated with that run.<jupyter_code>print(run.get_file_names())<jupyter_output>['azureml-logs/20_image_build_log.txt', 'azureml-logs/60_control_log.txt', 'azureml-logs/80_driver_log.txt', 'outputs/sklearn_mnist_model.pkl', 'driver_log', 'azureml-logs/azureml.log']<jupyter_text>Register the model in the workspace so that you (or other collaborators) can later query, examine, and deploy this model.<jupyter_code># register model 
model = run.register_model(model_name='sklearn_mnist', model_path='outputs/sklearn_mnist_model.pkl')
print(model.name, model.id, model.version, sep = '\t')<jupyter_output>Deprecated, use RunHistoryFacade.assets instead.<jupyter_text>Tidy up Compute Resource when not in use<jupyter_code>compute_target.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0837228114326802, 'fraction_numerical': 0.01842531008448679, 'mean_word_length': 5.243827160493828, 'pattern_counts': {'":': 0, '<': 81, '<?xml version=': 0, '>': 87, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '277652', 'license': 'MIT', 'n_tokens_mistral': 6618, 'n_tokens_neox': 6240, 'n_words': 2368, 'path': '#5 Training and Computer Vision Models on the Cloud using Azure ML Service.ipynb', 'repo_name': 'aribornstein/keras-cvworkshop', 'size': '22261'}