<jupyter_start><jupyter_text>Re-Purposing a Pretrained NetworkSince a large CNN is very time-consuming to train (even on a GPU), and requires huge amounts of data, is there any way to use a pre-calculated one instead of retraining the whole thing from scratch?This notebook shows how this can be done.  And it works surprisingly well.  How do we classify images with untrained classes?This notebook extracts a vector representation of a set of images using the GoogLeNet CNN pretrained on ImageNet.  It then builds a 'simple SVM classifier', allowing new images can be classified directly.  No retraining of the original CNN is required.<jupyter_code>import theano
import theano.tensor as T

import lasagne

import numpy as np
import scipy

import matplotlib.pyplot as plt
%matplotlib inline

import pickle
import time

CLASS_DIR='./images/cars'<jupyter_output><empty_output><jupyter_text>Functions for building the GoogLeNet model with Lasagne and preprocessing the images are defined in ```model.googlenet```.Build the model and select layers we need - the features are taken from the final network layer, before the softmax nonlinearity.<jupyter_code>from models.imagenet_theano import googlenet

cnn_layers = googlenet.build_model()
cnn_input_var = cnn_layers['input'].input_var
cnn_feature_layer = cnn_layers['loss3/classifier']
cnn_output_layer = cnn_layers['prob']

get_cnn_features = theano.function([cnn_input_var], lasagne.layers.get_output(cnn_feature_layer))

print("GoogLeNet Model defined")<jupyter_output><empty_output><jupyter_text>Load the pretrained weights into the network :<jupyter_code>import os
import urllib.request 

imagenet_theano = './data/imagenet_theano' 
googlenet_pkl = imagenet_theano+'/blvc_googlenet.pkl'

if not os.path.isfile(googlenet_pkl):
    if not os.path.exists(imagenet_theano):
        os.makedirs(imagenet_theano)
    print("Downloading GoogLeNet parameter file")
    urllib.request.urlretrieve(
        'https://s3.amazonaws.com/lasagne/recipes/pretrained/imagenet/blvc_googlenet.pkl', 
        googlenet_pkl)

params = pickle.load(open(googlenet_pkl, 'rb'), encoding='iso-8859-1')

model_param_values = params['param values']
imagenet_classes = params['synset words']

lasagne.layers.set_all_param_values(cnn_output_layer, model_param_values)

print("Loaded GoogLeNet params")<jupyter_output><empty_output><jupyter_text>Use the Network to create 'features' for the training imagesNow go through the input images and feature-ize them according to the pretrained network.NB: The pretraining was done on ImageNet - there wasn't anything specific to the recognition task we're doing here.<jupyter_code>import os
classes = sorted( [ d for d in os.listdir(CLASS_DIR) if os.path.isdir("%s/%s" % (CLASS_DIR, d)) ] )
classes # Sorted for for consistency
train = dict(f=[], features=[], target=[])

t0 = time.time()
for class_i,d in enumerate(classes):
    for f in os.listdir("%s/%s" % (CLASS_DIR, d,)):
        filepath = '%s/%s/%s' % (CLASS_DIR,d,f,)
        if os.path.isdir(filepath): continue
        im = plt.imread(filepath)
        rawim, cnn_im = googlenet.prep_image(im)

        prob = get_cnn_features(cnn_im)

        train['f'].append(filepath)
        train['features'].append(prob[0])
        train['target'].append( class_i )

        plt.figure()
        plt.imshow(rawim.astype('uint8'))
        plt.axis('off')

        plt.text(320, 50, '{}'.format(f), fontsize=14)
        plt.text(320, 80, 'Train as class "{}"'.format(d), fontsize=12)
    
print("DONE : %6.2f seconds each" %(float(time.time() - t0)/len(train),))<jupyter_output><empty_output><jupyter_text>Build an SVM model over the features<jupyter_code>#train['features'][0]
from sklearn import svm
classifier = svm.LinearSVC()
classifier.fit(train['features'], train['target']) # learn from the data<jupyter_output><empty_output><jupyter_text>Use the SVM model to classify the test set<jupyter_code>test_image_files = [f for f in os.listdir(CLASS_DIR) if not os.path.isdir("%s/%s" % (CLASS_DIR, f))]

t0 = time.time()
for f in sorted(test_image_files):
    im = plt.imread('%s/%s' % (CLASS_DIR,f,))
    rawim, cnn_im = googlenet.prep_image(im)
        
    prob = get_cnn_features(cnn_im)

    prediction_i = classifier.predict([ prob[0] ])
    decision     = classifier.decision_function([ prob[0] ])
                       
    plt.figure()
    plt.imshow(rawim.astype('uint8'))
    plt.axis('off')
                
    prediction = classes[ prediction_i[0] ]
                       
    plt.text(350, 50, '{} : Distance from boundary = {:5.2f}'.format(prediction, decision[0]), fontsize=20)
    plt.text(350, 75, '{}'.format(f), fontsize=14)
    
print("DONE : %6.2f seconds each" %(float(time.time() - t0)/len(test_image_files),))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11710581346716854, 'fraction_numerical': 0.011083228774571309, 'mean_word_length': 4.51036866359447, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '88936', 'license': 'MIT', 'n_tokens_mistral': 1645, 'n_tokens_neox': 1588, 'n_words': 406, 'path': 'notebooks/2-CNN/5-TransferLearning/2-ImageClassifier-googlenet_theano.ipynb', 'repo_name': 'So-AI-love/deep-learning-workshop', 'size': '4782'}