<jupyter_start><jupyter_text>Coursework 1: Image filteringIn this coursework we will explore some basic image filters used in computer vision. The corresponding lectures are Lectures 3 and 4 on image filtering and edge detection.What to do?* The coursework includes both coding questions and written questions. Please read both the text and code comment in this notebook to get an idea what you are supposed to implement.* First, run `jupyter-lab` or `jupyter-notebook` in the terminal to start the Jupyter notebook.* Then, complete and run the code to get the results.* Finally, please export (File | Export Notebook As...) or print (using the print function of your browser) the notebook as a pdf file, which contains your code, results and answers, and upload the pdf file onto Cate.Dependencies:* If you work on a college computer in the Computer Lab, where Ubuntu 18.04 is installed by default, you can use the following virtual environment for your work, where required Python packages are already installed.`source /vol/bitbucket/wbai/virt/computer_vision_2020/bin/activate`When you no longer need the virtual environment, you can exit it by running `deactivate`.* If you work on your own laptop using either Anaconda or plain Python, you can install new packages (such as numpy, imageio etc) running `conda install [package_name]` or `pip3 install [package_name]` in the terminal.<jupyter_code># Import libaries (provided)
import imageio
import numpy as np
import matplotlib.pyplot as plt
import noise
import scipy
import scipy.signal
import math
import time<jupyter_output><empty_output><jupyter_text>1. Moving average filter. (20 points)Read a specific input image and add noise to the image. Design a moving average filter of kernel size 3x3, 5x5 and 9x9 respectively. Apply the filter to the input image, show the filtering results and comment on the results.Design the kernel of the filter by yourself. Then, 2D image filtering can be performed using the function `scipy.signal.convolve2d()`.<jupyter_code># Read the image (provided)
image = imageio.imread('london.jpg')
print('Image dimension =', image.shape)
plt.imshow(image, cmap='gray')
plt.gcf().set_size_inches(12, 9)
# Corrupt the image with Gaussian noise (provided)
image_noisy = noise.add_noise(image, 'gaussian')
plt.imshow(image_noisy, cmap='gray')
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>**Note: from now on, please use the noisy image as the input for the filters.** 1.1 Filter the noisy image with a 3x3 moving average filter. (5 points)<jupyter_code># Design the filter h


# Convolve the corrupted image with h using scipy.signal.convolve2d function


# Print the filter (provided)
print('Filter h:')
print(h)

# Display the filtering result (provided)
plt.imshow(image_filtered, cmap='gray')
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>1.2 Filter the noisy image with a 5x5 moving average filter. (5 points)<jupyter_code># Design the filter h


# Convolve the corrupted image with h using scipy.signal.convolve2d function


# Print the filter (provided)
print('Filter h:')
print(h)

# Display the filtering result (provided)
plt.imshow(image_filtered, cmap='gray')
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>1.3 Filter the noisy image with a 9x9 moving average filter. (5 points)<jupyter_code># Design the filter h


# Convolve the corrupted image with h using scipy.signal.convolve2d function


# Print the filter (provided)
print('Filter h:')
print(h)

# Display the filtering result (provided)
plt.imshow(image_filtered, cmap='gray')
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>1.4 Comment on the filtering results. How do different kernel sizes influence the filtering results? (5 points)   2. Edge detection. (45 points)Perform edge detection using Sobel filters, as well as Gaussian + Sobel filters. Display the magnitude image of the Sobel filtering result and comment.  2.1 Implement 3x3 Sobel filters and convolve with the noisy image. (5 points)<jupyter_code># Design the Sobel filters


# Sobel filtering


# Calculate the gradient magnitude


# Print the filters (provided)
print('h_sobel_x:')
print(h_sobel_x)
print('h_sobel_y:')
print(h_sobel_y)

# Display the magnitude image (provided)
plt.imshow(sobel_mag, cmap='gray')
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>2.2 Design a 2D Gaussian filter. (8 points)<jupyter_code># Design the Gaussian filter
def gaussian_filter_2d(sigma):
    # sigma: the parameter sigma in the Gaussian kernel (unit: pixel)
    #
    # return: a 2D array for the Gaussian kernel
    
    h = ...
    return h

# Display the Gaussian filter when sigma = 3 pixel (provided)
sigma = 3
h = gaussian_filter_2d(sigma)
plt.imshow(h)<jupyter_output><empty_output><jupyter_text>2.3 Perform Gaussian smoothing ($\sigma$ = 3 pixels), followed by Sobel filtering. (5 points)<jupyter_code># Perform Gaussian smoothing before Sobel filtering


# Sobel filtering


# Calculate the gradient magnitude


# Display the magnitude image (provided)
plt.imshow(sobel_mag, cmap='gray', vmin=0, vmax=100)
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>2.4 Perform Gaussian smoothing ($\sigma$ = 7 pixels), followed by Sobel filtering. Evaluate the computational time for Gaussian smoothing. (7 points)<jupyter_code># Create the Gaussian filter


# Perform Gaussian smoothing


# Sobel filtering


# Calculate the gradient magnitude


# Display the magnitude image (provided)
plt.imshow(sobel_mag, cmap='gray', vmin=0, vmax=100)
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>2.5 Design 1D Gaussian filters along x-axis and y-axis respectively. (8 points)<jupyter_code># Design the Gaussian filter
def gaussian_filter_1d(sigma):
    # sigma: the parameter sigma in the Gaussian kernel (unit: pixel)
    #
    # return: a 1D array for the Gaussian kernel
   
    h = ...
    return h

# sigma = 7 pixel (provided)
sigma = 7

# The Gaussian filter along x-axis. Its shape is (1, sz).


# The Gaussian filter along y-axis. Its shape is (sz, 1).


# Display the filters (provided)
plt.subplot(1, 2, 1)
plt.imshow(h_x)
plt.subplot(1, 2, 2)
plt.imshow(h_y)<jupyter_output><empty_output><jupyter_text>2.6 Perform Gaussian smoothing (sigma = 7 pixels) using two separable filters, then apply the Sobel filters. Evaluate the computational time for separable Gaussian filtering. (7 points)<jupyter_code># Perform separable Gaussian smoothing before Sobel filtering


# Sobel filtering


# Calculate the gradient magnitude


# Display the magnitude image (provided)
plt.imshow(sobel_mag, cmap='gray', vmin=0, vmax=100)
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>2.7 Comment on the Gaussian + Sobel filtering results and the computational time. (5 points)   3. Laplacian filter. (20 points)Perform Laplacian filtering and Laplacian of Gaussian filtering. Display the results and comment. 3.1 Implement a 3x3 Laplacian filter. (5 points)<jupyter_code># Design the filter


# Laplacian filtering


# Print the filter (provided)
print('h:')
print(h)

# Display the filtering result (provided)
plt.imshow(lap, cmap='gray', vmin=-10, vmax=10)
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>3.2 Implement the Laplacian of Gaussian filter ($\sigma$ = 3 pixel). (10 points)<jupyter_code># Design the Gaussian filters
# The Gaussian filter along x-axis. Its shape is (1, sz).


# The Gaussian filter along y-axis. Its shape is (sz, 1).


# Gaussian smoothing


# Design the Laplacian filter


# Laplacian filtering


# Display the filtering result (provided)
plt.imshow(lap, cmap='gray', vmin=-10, vmax=10)
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output><jupyter_text>3.3 Comments on the filtering results. (5 points)   4. Exploration: Implement the 3x3 moving average filter using Pytorch. (15 points)[Pytorch](https://pytorch.org/) is a machine learning framework that supports filtering and convolution.The [Conv2D](https://pytorch.org/docs/stable/nn.htmlconv2d) operator takes an input array of dimension NxC1xXxY, applies the filter and outputs an array of dimension NxC2xXxY. Here, since we only have one image with one colour channel, we will set N=1, C1=1 and C2=1. You can read the documentation of Conv2D for more detail.<jupyter_code># Import libaries (provided)
import torch<jupyter_output><empty_output><jupyter_text>4.1 Expand the dimension of the noisy image into 1x1xXxY and convert it to a Pytorch tensor. (5 points)<jupyter_code># Expand the dimension of the numpy array


# Convert to a Pytorch tensor using torch.from_numpy<jupyter_output><empty_output><jupyter_text>4.2 Create a Pytorch Conv2D filter, set its kernel to be a 3x3 moving averaging filter. (5 points)<jupyter_code># Create the Conv2D filter (provided)
conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=3, padding=1, bias=False)

# Set the kernel weight<jupyter_output><empty_output><jupyter_text>4.3 Apply the filter to the noisy image tensor and display the output image. (5 points)<jupyter_code># Filtering


# Display the filtering result (provided)
plt.imshow(image_filtered, cmap='gray')
plt.gcf().set_size_inches(12, 9)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08992497320471597, 'fraction_numerical': 0.021543408360128618, 'mean_word_length': 5.479861111111111, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '300399', 'license': 'Apache-2.0', 'n_tokens_mistral': 3179, 'n_tokens_neox': 2859, 'n_words': 1074, 'path': 'coursework_1/.ipynb_checkpoints/coursework_1-checkpoint.ipynb', 'repo_name': 'Blair-Shi/computer_vision', 'size': '9330'}