<jupyter_start><jupyter_text>Resize and normalize input Calculate normalization parameters ( see https://github.com/ngeorgis/pytorch_onnx_openvino/issues/1 )<jupyter_code>cifar_mean   = (0.4914, 0.4822, 0.4465)
cifar_std    = (0.2023, 0.1994, 0.2010)
# use this based on 
# https://pytorch.org/docs/stable/torchvision/models.html
# and https://github.com/pytorch/examples/blob/42e5b996718797e45c46a25c55b031e6768f8440/imagenet/main.py#L89-L101
# also see https://github.com/pytorch/vision/issues/39

imagenet_mean = (0.485, 0.456, 0.406)
imagenet_std  = (0.229, 0.224, 0.225)
transform = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(imagenet_mean, imagenet_std),
])<jupyter_output><empty_output><jupyter_text>For CIFAR10 use this scale and mean for OpenVino model optimizer<jupyter_code>print("--scale_values=[",cifar_std[0]*255,",",cifar_std[1]*255,",",cifar_std[2]*255,"] ",
      "--mean_values=[",cifar_mean[0]*255,",",cifar_mean[1]*255,",",cifar_mean[2]*255,"]", sep="")<jupyter_output>--scale_values=[51.5865,50.847,51.255] --mean_values=[125.307,122.961,113.8575]<jupyter_text>For IMAGENET use this scale and mean for OpenVino model optimizer<jupyter_code>print("--scale_values=[",imagenet_std[0]*255,",",imagenet_std[1]*255,",",imagenet_std[2]*255,"] ",
      "--mean_values=[",imagenet_mean[0]*255,",",imagenet_mean[1]*255,",",imagenet_mean[2]*255,"]", sep="")

transform_test_IMAGENET = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])
transform_test = transform_test_IMAGENET<jupyter_output><empty_output><jupyter_text>First test ResNet18 in PyTorch<jupyter_code>net18 = resnet18(pretrained=True)
# set test mode for inference
net18.eval()
im = Image.open('dog.jpeg')
x = transform_test(im)
x = x.unsqueeze(dim=0)<jupyter_output>Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /home/omair/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth<jupyter_text>![alt text](dog.jpeg "Dog test image")  Run inference<jupyter_code>device = torch.device("cpu")
net18.to(device)
res18 = net18(x)[0]
res18.shape
print ("First ten output values: ", net18(x)[0][0:10])
values, indices = res18.max(0)
print (indices)
values, indices = torch.topk(res18, 10)
print (values)
print (indices)
number_top = 10
labels ="test_model.labels"
if labels:
    with open(labels, 'r') as f:
        labels_map = [x.split(sep=' ', maxsplit=1)[-1].strip() for x in f]
else:
    labels_map = None

print (labels_map)
print("Top {} results:\n===============\n".format(number_top))
for i, probs in enumerate(res18):
    if (i<number_top):
        print( labels_map[indices[i]], "\t\t", values[i], indices[i])<jupyter_output>Top 10 results:
===============

n02106662_German_shepherd,_German_shepherd_dog,_German_police_dog,_alsatian 		 tensor(19.9267, grad_fn=<SelectBackward>) tensor(235)
n02105162_malinois 		 tensor(17.6738, grad_fn=<SelectBackward>) tensor(225)
n02111129_Leonberg 		 tensor(14.8026, grad_fn=<SelectBackward>) tensor(255)
n02116738_African_hunting_dog,_hyena_dog,_Cape_hunting_dog,_Lycaon_pictus 		 tensor(13.8714, grad_fn=<SelectBackward>) tensor(275)
n02091467_Norwegian_elkhound,_elkhound 		 tensor(12.3885, grad_fn=<SelectBackward>) tensor(174)
n02105251_briard 		 tensor(12.3455, grad_fn=<SelectBackward>) tensor(226)
n02088094_Afghan_hound,_Afghan 		 tensor(11.3364, grad_fn=<SelectBackward>) tensor(160)
n02105412_kelpie 		 tensor(10.9077, grad_fn=<SelectBackward>) tensor(227)
n02113186_Cardigan,_Cardigan_Welsh_corgi 		 tensor(10.5278, grad_fn=<SelectBackward>) tensor(264)
n02112350_keeshond 		 tensor(10.4591, grad_fn=<SelectBackward>) tensor(261)<jupyter_text>Convert PyTorch to ONNX<jupyter_code>!pip3 install onnxruntime
!pip3 install onnxruntime-gpu
torch.onnx._export(net18, x, 'resnet18.onnx', export_params=True)
!ls -l resnet18.onnx
import time
start = time.time()
for i in range(0,10):
    y = net18(x)
end = time.time()
print(" Inference FPS = {}".format(1/((end - start)/10)))<jupyter_output>Inference FPS = 52.20289198493519<jupyter_text>Convert model to OpenVino<jupyter_code>mo_onnx.py --input_model resnet18.onnx --scale_values=[58.395,57.120,57.375] --mean_values=[123.675,116.28,103.53] --reverse_input_channels --disable_resnet_optimization --disable_fusing --disable_gfusing --data_type=FP32 --output_dir fp32
! ls -l fp16/ fp32<jupyter_output>fp16/:
total 22992
-rwxrwxrwx 1 omair omair 23396484 Mar  3 22:43 resnet18.bin
-rwxrwxrwx 1 omair omair    14574 Mar  3 22:43 resnet18.mapping
-rwxrwxrwx 1 omair omair   122963 Mar  3 22:43 resnet18.xml

fp32:
total 45840
-rwxrwxrwx 1 omair omair 46792928 Mar  3 22:43 resnet18.bin
-rwxrwxrwx 1 omair omair    14574 Mar  3 22:43 resnet18.mapping
-rwxrwxrwx 1 omair omair   123017 Mar  3 22:43 resnet18.xml
{'fraction_non_alphanumeric': 0.1518345834177985, 'fraction_numerical': 0.12325157105209811, 'mean_word_length': 6.213450292397661, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 34, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '194474', 'license': 'MIT', 'n_tokens_mistral': 2385, 'n_tokens_neox': 2002, 'n_words': 321, 'path': 'pytorch_onnx_openvino.ipynb', 'repo_name': 'aletheia-ai/pytorch_onnx_openvino', 'size': '4933'}