<jupyter_start><jupyter_text>This colab simply uses [KAIR](https://github.com/cszn/KAIR) as a base and contains some modifications and pre-configured cells. The goal was making the usage of USRNet userfriendly in colab.Here are examples: [Here](https://slow.pics/c/BZuyN6MG) and [here](https://slow.pics/c/JB5NY4eE).Notes:- All pictures have the same resolution. Smaller pictures were upscaled with nearest neighbour interpolation.- Using 4x scale on all tests if possible. - waifu2x-ncnn-vulkan only 2x because waifu2x-ncnn-vulkan can't 4x.- Using different config values than the original USRNet.- Using waifu2x default settings.- Gigapixel 4.4.5 without face stuff.Simple Tutorial:- Run cells with these play-buttons that are visible on the left side of the code/text. ```[ ]``` indicate a play-button.- Put your files in "input". That folder will appear with the path ```Google Drive/USRNet/input```. You can also create it yourself.- You can also input a .tar/.tar.gz file if you dont like using individial files to mitigate transfer time. Just run the correct cell for that. The default input path will be ```Google Drive/USRNet/input/data.tar``` or ```Google Drive/USRNet/input/data.tar.gz```.- After you are done, all files will be in ```Google Drive/USRNet/results```.Tip: You can split and merge images to mitigate VRAM with [IEU](https://github.com/ptrsuder/IEU.Winforms).<jupyter_code># Check GPU
!nvidia-smi
# Connect Google Drive
from google.colab import drive
drive.mount('/content/drive')
print('Google Drive connected.')
#@title Run this at startup
!git clone https://github.com/cszn/KAIR
%cd /content/KAIR/model_zoo
!pip install gdown
!gdown --id 1qz8aaYOAMhoKn07VppFjRsDflYpxeVmz
!gdown --id 1xPN26OW5YBN9-5QfK2BGnaovSutH7fLX
!gdown --id 1R5HKJzHJmou-3iUYd4cNLSyMeWXSOmD4
!gdown --id 1U4BV42Qf0dtBOVbhUNRay6di3j_ioZVM
%cd /content/KAIR
#!wget https://raw.githubusercontent.com/cszn/USRNet/master/main_test_realapplication.py
!mkdir /content/KAIR/testsets/set_real
#%cd /content/KAIR/testsets/set_real
#!wget https://raw.githubusercontent.com/cszn/USRNet/master/testsets/set_real/chip.png
# Creates folders in your Google Drive for input and output.
# You only need to apply this once.
!mkdir "/content/drive/My Drive/input"
!mkdir "/content/drive/My Drive/results"<jupyter_output><empty_output><jupyter_text>You can customize the python file yourself if you want, but the default will work as well. Interesting parameter:- model_name: Changes used model.- save_LE: If True, it will save a comparison.- noise_level_img: Configure noise level.- kernel_width_default_x1234: Configure Kernel size. Bigger Kernel will mean sharper.<jupyter_code>%%writefile /content/KAIR/main_test_realapplication.py
import os.path
import cv2
import logging
import glob
import time

import numpy as np
from datetime import datetime
from collections import OrderedDict
from scipy.io import loadmat
from scipy import ndimage
import scipy.io as scio

import torch

from utils import utils_deblur
from utils import utils_logger
from utils import utils_sisr as sr
from utils import utils_image as util

from models.network_usrnet import USRNet as net

'''
Spyder (Python 3.7)
PyTorch 1.4.0
Windows 10 or Linux

<NAME> (<EMAIL>)
github: https://github.com/cszn/USRNet
        https://github.com/cszn/KAIR

If you have any question, please feel free to contact with me.
<NAME> (e-mail: c<EMAIL>)

by <NAME> (12/March/2020)

# --------------------------------------------

testing code of USRNet for the Table 1 in the paper
@inproceedings{zhang2020deep,
  title={Deep unfolding network for image super-resolution},
  author={<NAME> and <NAME>, Luc and <NAME>},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
  pages={3217--3226},
  year={2020}
}
'''

def main():
    counter = 0
    time_values = []
    

    rootdir = '/content/KAIR/testsets/'
    files = glob.glob(rootdir + '/**/*.png', recursive=True)
    files_jpg = glob.glob(rootdir + '/**/*.jpg', recursive=True)
    files.extend(files_jpg)
    
    for f in files:
      start = time.time()
      counter += 1
      
      # ----------------------------------------
      # Preparation
      # ----------------------------------------
      model_name = 'usrnet'      # 'usrgan' | 'usrnet' | 'usrgan_tiny' | 'usrnet_tiny'
      testset_name = 'set_real'  # test set,  'set_real'
      test_image = f    # 'chip.png', 'comic.png'
      #test_image = 'comic.png'

      sf = 4                     # scale factor, only from {1, 2, 3, 4}
      show_img = False           # default: False
      save_E = True              # save estimated image
      save_LE = False             # save zoomed LR, Estimated images

      # ----------------------------------------
      # set noise level and kernel
      # ----------------------------------------
      if 'chip' in test_image:
          noise_level_img = 15       # noise level for LR image, 15 for chip
          kernel_width_default_x1234 = [0.6, 0.9, 1.7, 2.2] # Gaussian kernel widths for x1, x2, x3, x4
      else:
          noise_level_img = 15       # noise level for LR image, 0.5~3 for clean images
          kernel_width_default_x1234 = [0.4, 0.7, 1.5, 1.5] # default Gaussian kernel widths of clean/sharp images for x1, x2, x3, x4

      noise_level_model = noise_level_img/255.  # noise level of model
      kernel_width = kernel_width_default_x1234[sf-1]

      # set your own kernel width
      # kernel_width = 2.2

      k = utils_deblur.fspecial('gaussian', 25, kernel_width)
      k = sr.shift_pixel(k, sf)  # shift the kernel
      k /= np.sum(k)
      util.surf(k) if show_img else None
      # scio.savemat('kernel_realapplication.mat', {'kernel':k})

      # load approximated bicubic kernels
      #kernels = hdf5storage.loadmat(os.path.join('kernels', 'kernel_bicubicx234.mat'))['kernels']
      #kernels = loadmat(os.path.join('kernels', 'kernel_bicubicx234.mat'))['kernels']
      #kernel = kernels[0, sf-2].astype(np.float64)

      kernel = util.single2tensor4(k[..., np.newaxis])


      n_channels = 1 if 'gray' in  model_name else 3  # 3 for color image, 1 for grayscale image
      model_pool = 'model_zoo'  # fixed
      testsets = 'testsets'     # fixed
      results = 'results'       # fixed
      result_name = testset_name + '_' + model_name
      model_path = os.path.join(model_pool, model_name+'.pth')

      # ----------------------------------------
      # L_path, E_path
      # ----------------------------------------
      L_path = os.path.join(testsets, testset_name) # L_path, fixed, for Low-quality images
      E_path = os.path.join(results, result_name)   # E_path, fixed, for Estimated images
      util.mkdir(E_path)

      #logger_name = result_name
      #utils_logger.logger_info(logger_name, log_path=os.path.join(E_path, logger_name+'.log'))
      #logger = logging.getLogger(logger_name)

      device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

      # ----------------------------------------
      # load model
      # ----------------------------------------
      if 'tiny' in model_name:
          model = net(n_iter=6, h_nc=32, in_nc=4, out_nc=3, nc=[16, 32, 64, 64],
                      nb=2, act_mode="R", downsample_mode='strideconv', upsample_mode="convtranspose")
      else:
          model = net(n_iter=8, h_nc=64, in_nc=4, out_nc=3, nc=[64, 128, 256, 512],
                      nb=2, act_mode="R", downsample_mode='strideconv', upsample_mode="convtranspose")

      model.load_state_dict(torch.load(model_path), strict=True)
      model.eval()
      for key, v in model.named_parameters():
          v.requires_grad = False

      number_parameters = sum(map(lambda x: x.numel(), model.parameters()))
      #logger.info('Params number: {}'.format(number_parameters))
      model = model.to(device)
      #logger.info('Model path: {:s}'.format(model_path))

      #logger.info('model_name:{}, image sigma:{}'.format(model_name, noise_level_img))
      #logger.info(L_path)

      img = os.path.join(L_path, test_image)
      # ------------------------------------
      # (1) img_L
      # ------------------------------------
      img_name, ext = os.path.splitext(os.path.basename(img))
      img_L = util.imread_uint(img, n_channels=n_channels)
      img_L = util.uint2single(img_L)

      util.imshow(img_L) if show_img else None
      w, h = img_L.shape[:2]
      #logger.info('{:>10s}--> ({:>4d}x{:<4d})'.format(img_name+ext, w, h))

      # boundary handling
      boarder = 8     # default setting for kernel size 25x25
      img = cv2.resize(img_L, (sf*h, sf*w), interpolation=cv2.INTER_NEAREST)
      img = utils_deblur.wrap_boundary_liu(img, [int(np.ceil(sf*w/boarder+2)*boarder), int(np.ceil(sf*h/boarder+2)*boarder)])
      img_wrap = sr.downsample_np(img, sf, center=False)
      img_wrap[:w, :h, :] = img_L
      img_L = img_wrap

      util.imshow(util.single2uint(img_L), title='LR image with noise level {}'.format(noise_level_img)) if show_img else None

      img_L = util.single2tensor4(img_L)
      img_L = img_L.to(device)

      # ------------------------------------
      # (2) img_E
      # ------------------------------------
      sigma = torch.tensor(noise_level_model).float().view([1, 1, 1, 1])
      [img_L, kernel, sigma] = [el.to(device) for el in [img_L, kernel, sigma]]

      img_E = model(img_L, kernel, sf, sigma)

      img_E = util.tensor2uint(img_E)[:sf*w, :sf*h, ...]

      if save_E:
          util.imsave(img_E, os.path.join(E_path, img_name+'_x'+str(sf)+'_'+model_name+'.png'))

      # --------------------------------
      # (3) save img_LE
      # --------------------------------
      if save_LE:
          k_v = k/np.max(k)*1.2
          k_v = util.single2uint(np.tile(k_v[..., np.newaxis], [1, 1, 3]))
          k_factor = 3
          k_v = cv2.resize(k_v, (k_factor*k_v.shape[1], k_factor*k_v.shape[0]), interpolation=cv2.INTER_NEAREST)
          img_L = util.tensor2uint(img_L)[:w, :h, ...]
          img_I = cv2.resize(img_L, (sf*img_L.shape[1], sf*img_L.shape[0]), interpolation=cv2.INTER_NEAREST)
          img_I[:k_v.shape[0], :k_v.shape[1], :] = k_v
          util.imshow(np.concatenate([img_I, img_E], axis=1), title='LR / Recovered') if show_img else None
          util.imsave(np.concatenate([img_I, img_E], axis=1), os.path.join(E_path, img_name+'_x'+str(sf)+'_'+model_name+'_LE.png'))

      #loop_timer.update(end_time - start_time)
      end = time.time()
      final_time = end - start 
      time_values.append(final_time)
      print('Image %d out of %d' %(counter,len(files)))

      print(f"********** Time per frame (avg): %d s Time left: %d s **********" % ( (sum(time_values) / len(time_values)), (sum(time_values) / len(time_values)*(len(files)-counter))))



if __name__ == '__main__':

    main()

# Copy Data (individual pictures) to Colab, run upscaling and copy back results to Google Drive. If you use another
# model than usrnet, then you need to adjust the !cp path.
# Example: !cp /content/KAIR/results/set_real_usrgan/* "/content/drive/My Drive/USRNet/results/"
%cd /content/KAIR
!cp /content/drive/My\ Drive/USRNet/input/* /content/KAIR/testsets/set_real
!python main_test_realapplication.py
!cp /content/KAIR/results/set_real_usrnet/* "/content/drive/My Drive/USRNet/results/"
print("Done.")
# Copy .tar.gz to Colab, run upscaling, create .tar/.tar.gz and copy back archive to Google Drive.
# If you want to use the 7z GUI, here is how you do it. With the 7z GUI you can create a .tar, then create a new
# archive with only that file and select "gzip". You need to create an archive twice. The final file should be
# data.tar.gz. You can change between .tar/.tar.gz in the last two lines. If .tar.gz is used, then compression
# is applied.
%cd /content/KAIR
!cp "/content/drive/My Drive/USRNet/input/data.tar.gz" /content/KAIR/testsets/set_real/data.tar.gz
!tar -C /content/KAIR/testsets/set_real/ -zxvf /content/KAIR/testsets/set_real/data.tar.gz
!python main_test_realapplication.py
!tar -cvf /content/KAIR/results/set_real_usrnet/output.tar.gz /content/KAIR/results/set_real_usrnet/
!cp /content/KAIR/results/set_real_usrnet/output.tar.gz "/content/drive/My Drive/USRNet/results/output.tar.gz"
print("Done.")
# Copy .tar to Colab, run upscaling, create .tar/.tar.gz and copy back archive to Google Drive.
# You can change between .tar/.tar.gz in the last two lines. If .tar.gz is used, then compression is applied.
%cd /content/KAIR
!cp "/content/drive/My Drive/USRNet/input/data.tar" /content/KAIR/testsets/set_real/data.tar
!tar -C /content/KAIR/testsets/set_real/ -xvf /content/KAIR/testsets/set_real/data.tar
!python main_test_realapplication.py
!tar -cvf /content/KAIR/results/set_real_usrnet/output.tar.gz /content/KAIR/results/set_real_usrnet/
!cp /content/KAIR/results/set_real_usrnet/output.tar.gz "/content/drive/My Drive/USRNet/results/output.tar.gz"
print("Done.")
# [Optional] Clear upscaling folder in Colab
%cd /content/
!sudo rm -rf /content/KAIR/results/set_real_usrnet
!mkdir /content/KAIR/results/set_real_usrnet

!sudo rm -rf /content/KAIR/testsets
!mkdir /content/KAIR/testsets
!mkdir /content/KAIR/testsets/set_real
# [DEMO] USRNet demo
# Place files in /content/KAIR/testsets/set5 and results will be in /content/KAIR/results/set5_usrnet.
# This will create different low resolution files with different blur kernels and will show what
# USRNet can do. Creates 36 low resolution files and the upscaled image result folder.
%cd /content/KAIR
!python /content/KAIR/main_test_usrnet.py<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16381345926800472, 'fraction_numerical': 0.017045454545454544, 'mean_word_length': 2.8459137343927354, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 20, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '374117', 'license': 'MIT', 'n_tokens_mistral': 4791, 'n_tokens_neox': 4530, 'n_words': 1171, 'path': 'Colab-USRNet.ipynb', 'repo_name': 'styler00dollar/Colab-USRNet', 'size': '13599'}