<jupyter_start><jupyter_text>INF6804 Vision par ordinateurPolytechnique MontréalAuthor: <NAME>FlowNet optical flow estimator in inference.Original GitHub: https://github.com/NVIDIA/flownet2-pytorch Install required packages.<jupyter_code>%pip install -U torchvision==0.4.1 -f https://download.pytorch.org/whl/cu101/torch_stable.html scipy==1.1.0 scikit-image tensorboardX colorama tqdm setproctitle<jupyter_output><empty_output><jupyter_text>Clone the repo.<jupyter_code>import os
# get flownet2-pytorch source
if not os.path.exists('flownet2-pytorch'):
  !git clone https://github.com/NVIDIA/flownet2-pytorch.git<jupyter_output>Cloning into 'flownet2-pytorch'...
remote: Enumerating objects: 557, done.[K
Receiving objects:   0% (1/557)   
Receiving objects:   1% (6/557)   
Receiving objects:   2% (12/557)   
Receiving objects:   3% (17/557)   
Receiving objects:   4% (23/557)   
Receiving objects:   5% (28/557)   
Receiving objects:   6% (34/557)   
Receiving objects:   7% (39/557)   
Receiving objects:   8% (45/557)   
Receiving objects:   9% (51/557)   
Receiving objects:  10% (56/557)   
Receiving objects:  11% (62/557)   
Receiving objects:  12% (67/557)   
Receiving objects:  13% (73/557)   
Receiving objects:  14% (78/557)   
Receiving objects:  15% (84/557)   
Receiving objects:  16% (90/557)   
Receiving objects:  17% (95/557)   
Receiving objects:  18% (101/557)   
Receiving objects:  19% (106/557)   
Receiving objects:  20% (112/557)   
Receiving objects:  21% (117/557)   
Receiving objects:  22% (123/557)   
Receiving objects:  23% (129/557)   
Receiving objects:  24% (134/557)   
Receiving objects[...]<jupyter_text>Install the custom layers used in the repo.<jupyter_code>!cd flownet2-pytorch && bash install.sh<jupyter_output><empty_output><jupyter_text>Download the pre-trained weights.<jupyter_code>file_name = 'FlowNet2-S_checkpoint.pth.tar'
if not os.path.exists(file_name):
  !gdown 'https://drive.google.com/u/0/uc?export=download&confirm=m5go&id=1V61dZjFomwlynwlYklJHC-TLfdFom3Lg'<jupyter_output>Downloading...
From: https://drive.google.com/u/0/uc?export=download&confirm=m5go&id=1V61dZjFomwlynwlYklJHC-TLfdFom3Lg
To: /content/FlowNet2-S_checkpoint.pth.tar

0.00B [00:00, ?B/s]
4.72MB [00:00, 25.6MB/s]
17.3MB [00:00, 27.3MB/s]
34.1MB [00:00, 36.0MB/s]
50.9MB [00:00, 45.7MB/s]
62.4MB [00:00, 55.7MB/s]
71.3MB [00:01, 34.5MB/s]
101MB [00:02, 34.3MB/s] 
131MB [00:02, 46.6MB/s]
144MB [00:02, 48.8MB/s]
154MB [00:02, 57.4MB/s]
155MB [00:02, 55.8MB/s]<jupyter_text>Import libraries. In order for Colab to see our previously installed layers, we need to restart the runtime. Because of this, you might have to restart the execution from here. This phenomenon is discussed here: https://stackoverflow.com/questions/57831187/need-to-restart-runtime-before-import-an-installed-package-in-colab.<jupyter_code>%cd flownet2-pytorch
import torch
import numpy as np
import argparse
try:
  from models import FlowNet2S
  from utils.frame_utils import read_gen
except:
  os.kill(os.getpid(), 9)
%cd ..<jupyter_output>/content/flownet2-pytorch
/content<jupyter_text>Initialize the network.<jupyter_code>import sys
sys.argv = ['--rgb_max 255.0']
parser = argparse.ArgumentParser()
parser.add_argument("--rgb_max", type=float, default=255.)
args = parser.parse_args()
net = FlowNet2S(args).cuda()<jupyter_output><empty_output><jupyter_text>Load the pre-trained weights using Pytorch's state_dict.<jupyter_code>dict = torch.load("FlowNet2-S_checkpoint.pth.tar")
net.load_state_dict(dict["state_dict"])<jupyter_output><empty_output><jupyter_text>Load an image pair, convert them to pytorch tensors and feed them to the network.<jupyter_code>from skimage import io
import cv2
import os

IMG_BASE_DIR = 'https://raw.githubusercontent.com/gabilodeau/INF6804/master/images'
IMG_PAIR_NAMES = ['in000225.jpg', 'in000227.jpg']
img_pair = []
for img_name in IMG_PAIR_NAMES:
  img_pair.append(cv2.resize(io.imread(os.path.join(IMG_BASE_DIR, img_name)), (512, 512)))

tensor_pair = np.array(img_pair).transpose(3, 0, 1, 2)
tensor_pair = torch.from_numpy(tensor_pair.astype(np.float32)).unsqueeze(0).cuda()
result = net(tensor_pair)[0].squeeze()
data = result.data.cpu().numpy().transpose(1, 2, 0)<jupyter_output><empty_output><jupyter_text>Convert the optical flow numerical data to an rgb image, then display our image pair and our flow rgb representation.<jupyter_code>from utils.flow_utils import flow2img
from google.colab.patches import cv2_imshow

flow_image = flow2img(data)

imshow_size = (256, 256)
cv2_imshow(cv2.resize(img_pair[0], imshow_size))
cv2_imshow(cv2.resize(img_pair[1], imshow_size))
cv2_imshow(cv2.resize(flow_image, imshow_size))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13373083475298125, 'fraction_numerical': 0.08432708688245315, 'mean_word_length': 5.381793478260869, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '304293', 'license': 'MIT', 'n_tokens_mistral': 2063, 'n_tokens_neox': 1851, 'n_words': 374, 'path': 'flownet_example.ipynb', 'repo_name': 'gabilodeau/INF6804', 'size': '4706'}