<jupyter_start><jupyter_text>BNN on PynqThis notebook covers how to use Binary Neural Networks on Pynq. It shows an example of handwritten digit recognition using a binarized neural network composed of 4 fully connected layers with 1024 neurons each, trained on the MNIST dataset of handwritten digits. In order to reproduce this notebook, you will need an external USB Camera connected to the PYNQ Board. 1.Instantiate the classifierCreating a classifier will automatically download the correct bitstream onto the device and load the weights trained on the specified dataset. By default there is one set of weights for the LFC network, the MNIST<jupyter_code>import bnn
print(bnn.available_params(bnn.NETWORK_LFC))

classifier = bnn.PynqBNN(network=bnn.NETWORK_LFC)<jupyter_output><empty_output><jupyter_text>2. Download the network parametersThe parameters of the network are downloaded in the Programmable logic memory, storing the trained weights on the MNIST dataset.<jupyter_code>classifier.load_parameters("chars_merged")<jupyter_output><empty_output><jupyter_text>3. Load the image from the cameraThe image is captured from the external USB camera and stored locally. The image is then enhanced in contract and brightness to remove background noise. The resulting image should show the digit on a white background:<jupyter_code>from PIL import Image as PIL_Image
from PIL import ImageEnhance
from PIL import ImageOps

orig_img_path = '/home/xilinx/image.jpg'
!fswebcam  --no-banner --save {orig_img_path} -d /dev/video0 2> /dev/null # Loading the image from the webcam

img = PIL_Image.open(orig_img_path).convert("L")                          # convert in black and white  
  
#Image enhancement                  
contr = ImageEnhance.Contrast(img)  
img = contr.enhance(3)                                                    # The enhancement values (contrast and brightness)   
bright = ImageEnhance.Brightness(img)                                     # depends on backgroud, external lights etc  
img = bright.enhance(4.0)            

#img = img.rotate(180)                                                     # Rotate the image (depending on camera orientation)  
#Adding a border for future cropping  
img = ImageOps.expand(img,border=80,fill='white')   
threshold = 180    
img = img.point(lambda p: p > threshold and 255)   
img<jupyter_output><empty_output><jupyter_text>4. Crop and scale the image<jupyter_code>from PIL import Image as PIL_Image
import numpy as np
import math
from scipy import misc

#Find bounding box  
inverted = ImageOps.invert(img)  
box = inverted.getbbox()  
img_new = img.crop(box)  
width, height = img_new.size  
ratio = min((28./height), (28./width))  
background = PIL_Image.new('RGB', (28,28), (255,255,255))  
if(height == width):  
    img_new = img_new.resize((28,28))  
elif(height>width):  
    img_new = img_new.resize((int(width*ratio),28))  
    background.paste(img_new, (int((28-img_new.size[0])/2),int((28-img_new.size[1])/2)))  
else:  
    img_new = img_new.resize((28, int(height*ratio)))  
    background.paste(img_new, (int((28-img_new.size[0])/2),int((28-img_new.size[1])/2)))  
  
background  
img_data=np.asarray(background)  
img_data = img_data[:,:,0]  
misc.imsave('/home/xilinx/img_webcam.png', img_data)<jupyter_output><empty_output><jupyter_text>5. Convert to BNN input formatThe image is resized to comply with the MNIST standard. The image is resized at 28x28 pixels and the colors inverted.<jupyter_code>from array import *
from PIL import Image as PIL_Image
img_load = PIL_Image.open('/home/xilinx/img_webcam.png').convert("L")  
# Convert to BNN input format  
# The image is resized to comply with the MNIST standard. The image is resized at 28x28 pixels and the colors inverted.   
  
#Resize the image and invert it (white on black)  
smallimg = ImageOps.invert(img_load)  
smallimg = smallimg.rotate(0)  
  
data_image = array('B')  
  
pixel = smallimg.load()  
for x in range(0,28):  
    for y in range(0,28):  
        if(pixel[y,x] == 255):  
            data_image.append(255)  
        else:  
            data_image.append(1)  
          
# Setting up the header of the MNIST format file - Required as the hardware is designed for MNIST dataset         
hexval = "{0:#0{1}x}".format(1,6)  
header = array('B')  
header.extend([0,0,8,1,0,0])  
header.append(int('0x'+hexval[2:][:2],16))  
header.append(int('0x'+hexval[2:][2:],16))  
header.extend([0,0,0,28,0,0,0,28])  
header[3] = 3 # Changing MSB for image data (0x00000803)  
data_image = header + data_image  
output_file = open('/home/xilinx/image.images-idx3-ubyte', 'wb')  
data_image.tofile(output_file)  
output_file.close()   
smallimg<jupyter_output><empty_output><jupyter_text>6. Launching BNN in hardwareThe image is passed in the PL and the inference is performed<jupyter_code>class_out= classifier.inference("/home/xilinx/image.images-idx3-ubyte")
print("Class number: {0}".format(class_out))
print("Class name: {0}".format(classifier.class_name(class_out)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1025183422565933, 'fraction_numerical': 0.027166369224667856, 'mean_word_length': 3.8640308582449374, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 28, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '375581', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1709, 'n_tokens_neox': 1608, 'n_words': 468, 'path': 'notebooks/Char_Recognition.ipynb', 'repo_name': 'woodygzp/test1', 'size': '5043'}