<jupyter_start><jupyter_text>BNN on PynqThis notebook covers how to use Binary Neural Networks on Pynq. It shows an example of image recognition with a binarized neural inspired at VGG-16, featuring 6 convolutional layers, 3 Max Pool layers and 3 Fully connected layers 1. Instantiate a ClassifierCreating a classifier will automatically download the correct bitstream onto the device and load the weights trained on the specified dataset. By default there are three sets of weights to choose from - this example uses the StreetView house number set.<jupyter_code>from pynq import Overlay
import cffi 
ROOT_BNN = "/opt/python3.6/lib/python3.6/site-packages/bnn/" 
ol = Overlay(ROOT_BNN+"/bitstreams/cnv-pynq-pynq.bit")
ol.download()
ffi = cffi.FFI()
ROOT_BNN = "/opt/python3.6/lib/python3.6/site-packages/bnn/" 
ffi.cdef("""
void load_parameters(const char* path);
unsigned int inference(const char* path, unsigned int results[64], int number_class, float *usecPerImage);
unsigned int* inference_multiple(const char* path, int number_class, int *image_number, float *usecPerImage, unsigned int enable_detail);
void free_results(unsigned int * result);
void deinit();
"""
)
NN_hw=ffi.dlopen(ROOT_BNN+"/libraries/python_hw-cnv-pynq.so")
NN_sw=ffi.dlopen(ROOT_BNN+"/libraries/python_sw-cnv-pynq.so")
ol.bitstream.timestamp<jupyter_output><empty_output><jupyter_text>2. Download the network parametersThe parameters of the network are downloaded in the Programmable logic memory, storing the trained weights on the Street view house number dataset.<jupyter_code>NN_hw.load_parameters(bytes(ROOT_BNN+"/params/streetview", encoding='utf-8'))<jupyter_output><empty_output><jupyter_text>3. Open image to be classifiedThe image to be classified is loaded, showd and resized to meet the BNN requirements (scaled to 32x32 pixels)<jupyter_code>from PIL import Image
import numpy as np
im = Image.open('/home/xilinx/jupyter_notebooks/bnn/6.png')
# We resize the downloaded image to be 32x32 pixels as expected from the BNN
im
im.thumbnail((32, 32), Image.ANTIALIAS)
background = Image.new('RGBA', (32, 32), (0, 0, 0, 0))
background.paste(
    im, (int((32 - im.size[0]) / 2), int((32 - im.size[1]) / 2))
)
# We write the image into the format used in the Cifar-10 dataset for code compatibility 
im = (np.array(background))
r = im[:,:,0].flatten()
g = im[:,:,1].flatten()
b = im[:,:,2].flatten()
label = [1]

out = np.array(list(label) + list(r) + list(g) + list(b),np.uint8)
out.tofile("/home/xilinx/out.bin")
background<jupyter_output><empty_output><jupyter_text>4. Loading classes descriptionWe load the class description, that links the class identified from the BNN and the actual meaning of that class<jupyter_code>import os
with open (os.path.join(ROOT_BNN+"/params/streetview/", "classes.txt")) as f:
    classes = [c.strip() for c in f.readlines()]<jupyter_output><empty_output><jupyter_text>5. Launching BNN in hardwareThe image is passed in the PL and the inference is performedNOTE: To execute the next block, you will need to have install wurlitzer (pip3.6 install wurlitzer)<jupyter_code>from wurlitzer import pipes
with pipes() as (stdout, stderr):
    class_out=NN_hw.inference(b"/home/xilinx/out.bin",ffi.NULL,10, ffi.NULL) 
print(stdout.read())    
print("Identified number: {0}".format(classes[class_out]))<jupyter_output>Packing and interleaving CIFAR-10 inputs...
Running prebuilt CIFAR-10 test for 1 images...
Inference took 1615 microseconds, 1615 usec per image
Classification rate: 619.195 images per second

Identified number: 6<jupyter_text>6. Launching BNN in softwareThe inference on the same image is performed in sofware on the ARM core<jupyter_code>NN_sw.load_parameters(bytes(ROOT_BNN+"/params/streetview", encoding='utf-8'))

with pipes() as (stdout, stderr):
    class_out=NN_sw.inference(b"/home/xilinx/out.bin",ffi.NULL,10, ffi.NULL) 
print(stdout.read())    
print("Identified number: {0}".format(classes[class_out]))<jupyter_output>Packing and interleaving CIFAR-10 inputs...
Running prebuilt CIFAR-10 test for 1 images...
Inference took 811819 microseconds, 811819 usec per image
Classification rate: 1.2318 images per second

Identified number: 6
{'fraction_non_alphanumeric': 0.1116435074269286, 'fraction_numerical': 0.02635361763296598, 'mean_word_length': 5.701444622792938, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '151174', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1465, 'n_tokens_neox': 1375, 'n_words': 426, 'path': 'notebooks/SVHN-cffi.ipynb', 'repo_name': 'mikucy/BNN', 'size': '4174'}