<jupyter_start><jupyter_text>(image-filtering:convolution)= ConvolutionWhen we apply a so called _linear_ filter to an image we compute every new pixel as a weighted sum of its neighbors. The process is called [_convolution_](https://en.wikipedia.org/wiki/Convolution) and the matrix defining the weights is called _convolution kernel_. In the context of microscopy, we often speal about the point-spread-function (PSF) of microscopes. This PSF technically describes how an image is convolved by the microsocpe before we save it to disk.<jupyter_code>import numpy as np
import pyclesperanto_prototype as cle
from skimage.io import imread
from pyclesperanto_prototype import imshow
from skimage import filters
from skimage.morphology import ball
from scipy.ndimage import convolve
import matplotlib.pyplot as plt

cle.select_device('RTX')<jupyter_output><empty_output><jupyter_text>For demonstrating the principle of convolution, we first define an example image that's rather simple.<jupyter_code>image = np.asarray([
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  [0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  [0, 0, 0, 0, 0, 0, 0, 2, 0, 0],
  [0, 0, 0, 0, 0, 0, 1, 0, 0, 0],
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
]).astype(float)

imshow(image)<jupyter_output><empty_output><jupyter_text>Next, we define a simple convolution kernel, that is represented by a small image.<jupyter_code>kernel = np.asarray([
  [0, 1, 0],
  [1, 1, 1],
  [0, 1, 0],
])<jupyter_output><empty_output><jupyter_text>Next, we convolve the image with the kernel using [scipy.ndimage.convolve](https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.convolve.html). When we print out the result, we can see how a `1` in the original image spreads, because for every directly neighboring pixel, the kernel sums the neighbor intensities. If there are multuple pixels with intensity > 0 in the original image, the resulting image will in their neighborhood compute the sum. You could call this kernel a local sum-kernel.<jupyter_code>convolved = convolve(image, kernel)

imshow(convolved, colorbar=True)
convolved<jupyter_output><empty_output><jupyter_text>Other kernelsDepending on which kerne is used for the convolution, the images can look quite differently. A _mean_-kernel for example computes the average pixel intensity locally:<jupyter_code>mean_kernel = np.asarray([
  [0, 0.2, 0],
  [0.2, 0.2, 0.2],
  [0, 0.2, 0],
])
mean_convolved = convolve(image, mean_kernel)

imshow(mean_convolved, colorbar=True)
mean_convolved<jupyter_output><empty_output><jupyter_text>The following kernel is a simple form of a [Laplace operator](https://en.wikipedia.org/wiki/Laplace_operator).<jupyter_code>laplace_operator = np.asarray([
  [0, 1, 0],
  [1, -4, 1],
  [0, 1, 0],
])
laplacian = convolve(image, laplace_operator)

imshow(laplacian, colorbar=True)
laplacian<jupyter_output><empty_output><jupyter_text>To demonstrate what these different kernels do, we apply them to the MRI image shown before.<jupyter_code># open dataset and extract single plane
noisy_mri = imread('../../data/Haase_MRT_tfl3d1.tif')[90].astype(float)

# zoom in by cropping a part out
noisy_mri_zoom = noisy_mri[50:100, 50:100]
convolved_mri = convolve(noisy_mri_zoom, kernel)
mean_mri = convolve(noisy_mri_zoom, mean_kernel)
laplacian_mri = convolve(noisy_mri_zoom, laplace_operator)
fig, axes = plt.subplots(2, 2, figsize=(10,10))

imshow(noisy_mri_zoom, plot=axes[0,0])
axes[0,0].set_title("original")
imshow(laplacian_mri, plot=axes[0,1])
axes[0,1].set_title("Laplacian")
imshow(convolved_mri, plot=axes[1,0])
axes[1,0].set_title("sum-kernel")
imshow(mean_mri, plot=axes[1,1])
axes[1,1].set_title("Mean-kernel")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12348922753547031, 'fraction_numerical': 0.04203888596952181, 'mean_word_length': 5.690685413005272, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 30, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '245171', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1550, 'n_tokens_neox': 1371, 'n_words': 433, 'path': 'docs/18_image_filtering/convolution.ipynb', 'repo_name': 'rayanirban/BioImageAnalysisNotebooks', 'size': '3806'}