<jupyter_start><jupyter_text>Convolutional: Instant Recognition with Caffe  1. Setup* First, set up Python, `numpy`, and `matplotlib`.<jupyter_code>import time
# set up Python environment: numpy for numerical routines, and matplotlib for plotting
import numpy as np
import matplotlib.pyplot as plt
# display plots in this notebook
%matplotlib inline

# set display defaults
plt.rcParams['figure.figsize'] = (10, 10)        # large images
plt.rcParams['image.interpolation'] = 'nearest'  # don't interpolate: show square pixels
plt.rcParams['image.cmap'] = 'gray'  # use grayscale output rather than a (potentially misleading) color heatmap<jupyter_output><empty_output><jupyter_text>* Load `caffe`.<jupyter_code># The caffe module needs to be on the Python path;
#  we'll add it here explicitly.
import sys
caffe_root = '../'  # this file should be run from {caffe_root}/examples (otherwise change this line)
sys.path.insert(0, caffe_root + 'python')

import caffe
# If you get "No module named _caffe", either you have not built pycaffe or you have the wrong path.<jupyter_output><empty_output><jupyter_text>* If needed, download the reference model ("CaffeNet", a variant of AlexNet).  2. Load net and set up input preprocessing* Set Caffe to CPU mode and load the net from disk.<jupyter_code>caffe.set_mode_cpu()

model_def = caffe_root + 'models/DCGAN-finetune/deploy_DCGAN_train_val.prototxt'
model_weights = caffe_root + 'models/DCGAN-finetune/model-Licmophora/_iter_1500.caffemodel'

net = caffe.Net(model_def,      # defines the structure of the model
                model_weights,  # contains the trained weights
                caffe.TEST)     # use test mode (e.g., don't perform dropout)<jupyter_output><empty_output><jupyter_text>* Set up input preprocessing. (We'll use Caffe's `caffe.io.Transformer` to do this, but this step is independent of other parts of Caffe, so any custom preprocessing code may be used).    Our default CaffeNet is configured to take images in BGR format. Values are expected to start in the range [0, 255] and then have the mean ImageNet pixel value subtracted from them. In addition, the channel dimension is expected as the first (_outermost_) dimension.        As matplotlib will load images with values in the range [0, 1] in RGB format with the channel as the _innermost_ dimension, we are arranging for the needed transformations here.<jupyter_code># create transformer for the input called 'data'
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})

transformer.set_transpose('data', (2,0,1))  # move image channels to outermost dimension
#transformer.set_raw_scale('data', 255)      # rescale from [0, 1] to [0, 255]<jupyter_output><empty_output><jupyter_text>3. CPU classification* Now we're ready to perform classification. Even though we'll only classify one image, we'll set a batch size of 50 to demonstrate batching.<jupyter_code># set the size of the input (we can skip this if we're happy
#  with the default; we can also change it later, e.g., for different batch sizes)
net.blobs['data'].reshape(100,        # batch size
                          1,         # 1-channel images
                          64, 64)  # image size is 227x227
net.forward()
plt.imshow(net.blobs['data'].data[:8, 0].transpose(2, 0, 1).reshape(64, 8*64), cmap='gray'); plt.axis('off')
print 'train labels:', net.blobs['label'].data[:8]<jupyter_output><empty_output><jupyter_text>* Load an image (that comes with Caffe) and perform the preprocessing we've set up.<jupyter_code>image = caffe.io.load_image(caffe_root + 'data/MVCO/IFCB1_2012_258_172757_00311.png')
transformed_image = transformer.preprocess('data', image)
plt.imshow(image)<jupyter_output><empty_output><jupyter_text>* Adorable! Let's classify it!<jupyter_code># copy the image data into the memory allocated for the net
net.blobs['data'].data[...] = transformed_image.reshape(3,1,64,64) # reshape the imagedata and fit int the network input

### perform classification
output = net.forward()

#output_prob = output['loss']  # the output probability vector for the first image in the batch

#print 'predicted class is:', output_prob.argmax()<jupyter_output><empty_output><jupyter_text>* The net gives us a vector of probabilities; the most probable class was the 281st one. But is that correct? Let's check the ImageNet labels...  5. Examining intermediate output* A net is not just a black box; let's take a look at some of the parameters and intermediate activations.First we'll see how to read out the structure of the net in terms of activation and parameter shapes.* For each layer, let's look at the activation shapes, which typically have the form `(batch_size, channel_dim, height, width)`.    The activations are exposed as an `OrderedDict`, `net.blobs`. * Since we're dealing with four-dimensional data here, we'll define a helper function for visualizing sets of rectangular heatmaps.<jupyter_code>def vis_square(data, label, i):
    """Take an array of shape (n, height, width) or (n, height, width, 3)
       and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n)"""
    
    # normalize data for display
    data = (data - data.min()) / (data.max() - data.min())
    
    # force the number of filters to be square
    n = int(np.ceil(np.sqrt(data.shape[0])))
    padding = (((0, n ** 2 - data.shape[0]),
               (0, 1), (0, 1))                 # add some space between filters
               + ((0, 0),) * (data.ndim - 3))  # don't pad the last dimension (if there is one)
    data = np.pad(data, padding, mode='constant', constant_values=1)  # pad with ones (white)
    
    # tile the filters into an image
    data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1)))
    data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:])
    plt.imshow(data); plt.axis('off')
    name = str(i)+label + '.png'
    plt.imsave(name,data)<jupyter_output><empty_output><jupyter_text>* First we'll look at the first layer filters, `conv1`<jupyter_code># the parameters are a list of [weights, biases]
filters = net.params['conv1'][0].data
#print filters.shape
vis_square(filters)<jupyter_output><empty_output><jupyter_text>* The first layer output, `conv1` (rectified responses of the filters above, first 36 only)<jupyter_code>filters = net.params['gen_conv5'][0].data[1]
print filters.shape
vis_square(filters)
output = net.forward()
feat = net.blobs['gen_conv2'].data[1]
vis_square(feat)

numbers = 100
for i in range(0,numbers):
    output = net.forward()
    feat = net.blobs['gen_conv5'].data[i]
    vis_square(feat, '_Licmophora62_1k_generate', i)<jupyter_output><empty_output><jupyter_text>* The fifth layer after pooling, `pool5`<jupyter_code>output = net.forward()
feat = net.blobs['gen_conv5'].data[:8,0]
#vis_square(feat)
plt.imshow(feat.transpose(2, 0, 1).reshape(64,8*64), cmap='gray'); plt.axis('off')<jupyter_output><empty_output><jupyter_text>* The first fully connected layer, `fc6` (rectified)    We show the output values and the histogram of the positive values<jupyter_code>#print net.params['ip1encode'][0].data
feat = net.blobs['rand_vec'].data[0]
plt.subplot(2, 1, 1)
plt.plot(feat.flat)
plt.subplot(2, 1, 2)
_ = plt.hist(feat.flat[feat.flat > 0], bins=100)
feat = net.blobs['fc6decode'].data[0]
plt.subplot(2, 1, 1)
plt.plot(feat.flat)
plt.subplot(2, 1, 2)
_ = plt.hist(feat.flat[feat.flat > 0], bins=100)<jupyter_output><empty_output><jupyter_text>* The final probability output, `prob`<jupyter_code>feat = net.blobs['prob'].data[0]
plt.figure(figsize=(15, 3))
plt.plot(feat.flat)<jupyter_output><empty_output><jupyter_text>Note the cluster of strong predictions; the labels are sorted semantically. The top peaks correspond to the top predicted labels, as shown above.  6. Extract the featureTo extract deepfeature<jupyter_code>#load the net and set caffe model
import sys
caffe_root = '../'  # this file should be run from {caffe_root}/examples (otherwise change this line)
sys.path.insert(0, caffe_root + 'python')
caffe.set_mode_cpu()

model_def = caffe_root + 'models/CAE_mvco/CAE_vgg_train_val.prototxt'
model_weights = caffe_root + 'models/CAE_mvco/_iter_120000.caffemodel'

net = caffe.Net(model_def,      # defines the structure of the model
                model_weights,  # contains the trained weights
                caffe.TEST)     # use test mode (e.g., don't perform dropout)

# perform classification
f = open('feature.txt', "a+")
iteration = 2
for a in range(0,iteration):
    net.forward()
    f.write(net.blobs['fc6decode'].data[0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11812391430225826, 'fraction_numerical': 0.02119281991893457, 'mean_word_length': 4.618737800910865, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '147437', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 2953, 'n_tokens_neox': 2760, 'n_words': 898, 'path': 'python/Generative_new_plankton.ipynb', 'repo_name': 'godisboy/caffe_GAN', 'size': '8635'}