<jupyter_start><jupyter_text>Image Segmentation with Pre-trained Models We can often leverage pre-trained models contributed by the community. The [MXNet Model Zoo](http://mxnet.io/model_zoo/) contains fast implementations of many state-of-the-art models, with pre-trained weights included.In this tutorial, we demonstrate how to use a pre-trained network and perform prediction on a new image.The task here is image segmentation, where the network learns to assign each pixel to a category, such as shown below:Here we use a pre-trained segmentation model called FCN-xs (based on [this](https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf) publication). This is a convolutional neural network that was trained on the PASCAL VOC 2011 dataset, which includes 2,207 images similar to those above. Each image was annotated to segment twenty classes of objects: `person, bird, cat, cow, dog, horse, sheep, aeroplane, bicycle, boat, bus, car, motorbike, train, bottle, chair, dining table, potted plant, and tv monitor`.  Prepare the Data The required model files are hosted at https://bitbucket.org/krishnasumanthm/mxnet_image_segmentationWe first define a download function and download pre-trained model, symbol file and a test image.Note: The pre-trained model is about 500MB, so first time downloading might take some time.<jupyter_code>import logging
import numpy as np
import mxnet as mx
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import os, urllib2, time
import warnings
warnings.simplefilter('ignore',DeprecationWarning)
import sys
from PIL import Image
import utils<jupyter_output><empty_output><jupyter_text>Load the modelWe first load the model by calling the `mx.model.load_checkpoint()` method. This loads the network definiton, which based on a `json` file, and the pre-trained weights.<jupyter_code>model, params, states = mx.model.load_checkpoint('data/FCN8s_VGG16', epoch=19)<jupyter_output><empty_output><jupyter_text>The model's weights are stored in the `params` dictionary, and easy to inspect: For example, here we compute the average magnitude of the weights in the first convolutional layer:<jupyter_code>params['conv1_1_weight']  # this is an array of size (64, 3, 3, 3)

magnitude = np.mean(np.abs(params['conv1_1_weight'].asnumpy()))
print "First layer weights have average magnitude of: {}".format(magnitude)<jupyter_output><empty_output><jupyter_text>Preprocess the data for evaluationBelow we create a helper function to load the image, perform some minor preprocessing, and then convert the image to a numpy array.<jupyter_code>def load_image(img_path):
    # Function to convert input image to np.array
    """get the (1, 3, h, w) np.array data for the img_path"""
    mean = np.array([123.68, 116.779, 103.939])  # (R,G,B)
    
    img = Image.open(img_path)
    img.thumbnail((800, 800), Image.ANTIALIAS)
    img = np.array(img, dtype=np.float32)
    reshaped_mean = mean.reshape(1, 1, 3)
    img = img - reshaped_mean
    img = np.swapaxes(img, 0, 2)
    img = np.swapaxes(img, 1, 2)
    img = np.expand_dims(img, axis=0)
    return img<jupyter_output><empty_output><jupyter_text>Evaluate the modelThen, we create a function below that:1. Tells the model what the input image size to expect.2. Creates an executor, while binding the loaded parameters and states3. Runs the inference pass via `forward()` method.4. Saves the images and display in notebook.<jupyter_code>def predict(img_path, model, params, states):
    img = load_image(img_path)
    
    # Input images are stored in a 4-D matrix 
    params["data"] = mx.nd.array(img, mx.cpu()) 
    data_shape = params["data"].shape
    
    # Output image has the same number of pixels
    label_shape = (1, data_shape[2]*data_shape[3])
    params["softmax_label"] = mx.nd.empty(label_shape, mx.cpu())
    
    # create an 'executor' and bind the parameters and states
    executor = model.bind(mx.cpu(), params, aux_states=states)

    tic = time.time()

    # run the inference pass
    executor.forward(is_train=False)
    print "Time taken for forward pass: {:.3f} milli sec".format((time.time()-tic)*1000)

    # save the images and print the results
    print "Saving images..."
    utils.print_images(executor.outputs[0], img_path)


predict("person_bicycle.jpg", model, params, states)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09350529709811146, 'fraction_numerical': 0.017273146015660986, 'mean_word_length': 4.782956058588549, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '515831', 'license': 'Apache-2.0', 'n_tokens_mistral': 1375, 'n_tokens_neox': 1300, 'n_words': 484, 'path': 'Part_3_Segmentation.ipynb', 'repo_name': 'halalyon/mxnet_workshop', 'size': '4342'}