<jupyter_start><jupyter_text>This runs in a standard anaconda environment, which was created by the following commands`conda create -n onnx python=3 anaconda``source activate onnx``conda install -c caffe2 caffe2``pip install git+git://github.com/onnx/onnx.git@master`  load the fixed batchnorm onnx file<jupyter_code>PATH = "data/project/"
model = onnx.load("sleeves_fixed_batchnorm.proto")

onnx.checker.check_model(model)
onnx.helper.printable_graph(model.graph)<jupyter_output><empty_output><jupyter_text>rep is the caffe2 object, calling run with a numpy array will predict on that array<jupyter_code>rep = backend.prepare(model)

outputs = rep.run(np.random.randn(1, 3, 224, 224).astype(np.float32))

outputs<jupyter_output><empty_output><jupyter_text>great, it produces output on random input<jupyter_code>test_img_numpy = np.load("test_img_numpy.npy")
test_label_numpy = np.load("test_label_numpy.npy")
test_img_numpy[:1].shape
outputs = rep.run(test_img_numpy[:1])

np.set_printoptions(suppress=True)
np.exp((outputs[0]))<jupyter_output><empty_output><jupyter_text>exactly the same output on the saved dataset from fastai<jupyter_code>import math
def preprocess(im,sz=224):
    h,w,_ = im.shape
    if h >= w:
        im = cv2.resize(im, (sz, math.floor(h/w * sz)))
    else:
        im = cv2.resize(im, (math.floor(w/h * sz), sz))
    
    #im = center_crop(im, sz)
    im = center_crop(im,sz)
    #im = cv2.resize(im, sz)
    
    
    im = im.astype(np.float32) / 255
    im = im[:,:,::-1]
    im = np.transpose(im, [2,0,1])
    return normalize(im).astype(np.float32)

def center_crop(im,sz):
    h,w,_ = im.shape
    if h >= w:
        y_center = math.floor(h/2)
        ymin = int(y_center - sz/2)
        ymax = int(y_center + sz/2)
        
        im = im[ymin:ymax, 0:sz]
    
    else:
        x_center = math.floor(w/2)
        xmin = int(x_center - sz/2)
        xmax = int(x_center + sz/2 )
        im = im[0:sz, xmin:xmax]
    return im

def normalize(im):
    mean = np.array([0.485, 0.456, 0.406]).reshape((3,1,1))
    std = np.array([0.229, 0.224, 0.225]).reshape((3,1,1))
    return (im - mean)/std

def denorm(im):
    #just for displaying purposes
    mean = np.array([0.485, 0.456, 0.406]).reshape((3,1,1))
    std = np.array([0.229, 0.224, 0.225]).reshape((3,1,1))
    return (im *std) + mean<jupyter_output><empty_output><jupyter_text>image preprocessing. passing a single image read by opencv into preprocess will return a correctly preprocessed image, doesnt produce the exact same results as fastai image preprocessing (yet)<jupyter_code>test_img = cv2.imread(PATH+ "test/160306600.jpg")
test_arr = preprocess(test_img)
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>this preprocessing<jupyter_code>plt.imshow(np.clip(denorm(test_arr).transpose(1,2,0),0,1))
test_img2 = cv2.imread(PATH + "test/1.jpg")
test_arr2 = preprocess(test_img2)
plt.imshow(np.clip(denorm(test_arr2).transpose(1,2,0),0,1))<jupyter_output><empty_output><jupyter_text>fastai preprocessing<jupyter_code>plt.imshow(np.clip(denorm(test_img_numpy[0]).transpose(1,2,0),0,1))
plt.imshow(np.clip(denorm(test_img_numpy[6]).transpose(1,2,0),0,1))
test_arr.dtype
test_arr.shape
np.exp(rep.run(np.expand_dims(test_arr,0)))<jupyter_output><empty_output><jupyter_text>predictions are close, but not exactly the same as the images are slightly different<jupyter_code># images are not identical
np.allclose(test_img_numpy[0], test_arr)
((test_img_numpy[0] == test_arr) == True).sum()<jupyter_output><empty_output><jupyter_text>Extra: without the batchnorm fix<jupyter_code>model_no_batchnorm_fix = onnx.load("sleeves_no_batchnorm_fix.proto")

onnx.checker.check_model(model_no_batchnorm_fix)
onnx.helper.printable_graph(model_no_batchnorm_fix.graph)<jupyter_output><empty_output><jupyter_text>rep is the caffe2 object, calling run with a numpy array will predict on that array<jupyter_code>rep_no_batchnorm_fix = backend.prepare(model_no_batchnorm_fix)

outputs_no_batchnorm_fix = rep_no_batchnorm_fix.run(np.random.randn(1, 3, 224, 224).astype(np.float32))

outputs_no_batchnorm_fix<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13794772507260405, 'fraction_numerical': 0.037270087124878996, 'mean_word_length': 5.309923664122137, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '325566', 'license': 'Apache-2.0', 'n_tokens_mistral': 1688, 'n_tokens_neox': 1622, 'n_words': 281, 'path': 'courses/dl1/onnx-import-project.ipynb', 'repo_name': 'jaidmin/fastai-caffe2', 'size': '4132'}