<jupyter_start><jupyter_text>Fundamentals of image processing using scikit image NumPy is the fundamental package for scientific computing with Python. It contains among other things:- a powerful N-dimensional array object- sophisticated (broadcasting) functions- tools for integrating C/C++ and Fortran code- useful linear algebra, Fourier transform, and random number capabilitieshttps://numpy.org/ Scikit-image is a collection of algorithms for image processing. It contains:- algorithms for image filtering, registration, and segmentation amoungst others- great tutorials and examples galleryhttps://scikit-image.org Before we can doing image processing with `scikit-image`, we need to understand how images are represented. For this, we will use NumPy and material from their introductory tutorial here - https://numpy.org/devdocs/user/quickstart.html  Introduction to NumPy NumPy’s main object is the homogeneous multidimensional array. It is a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers. In NumPy dimensions are called axes.For example, the coordinates of a point in 3D space [1, 2, 1] has one axis. That axis has 3 elements in it, so we say it has a length of 3.In the example pictured below, the array has 2 axes. The first axis has a length of 2, the second axis has a length of 3.```[[ 1., 0., 0.], [ 0., 1., 2.]] ``` NumPy’s array class is called `ndarray`. The more important attributes of an ndarray object are:- `ndarray.ndim`the number of axes (dimensions) of the array.- `ndarray.shape`the dimensions of the array. This is a tuple of integers indicating the size of the array in each dimension. For a matrix with n rows and m columns, shape will be (n,m). The length of the shape tuple is therefore the number of axes, ndim.- `ndarray.size`the total number of elements of the array. This is equal to the product of the elements of shape.- `ndarray.dtype`an object describing the type of the elements in the array. One can create or specify dtype’s using standard Python types. Additionally NumPy provides types of its own. numpy.int32, numpy.int16, and numpy.float64 are some examples.   An example with a random array First we import NumPy and then we can create a random 2D array and inspect some of its propteries<jupyter_code>import numpy as np
a = np.random.random((3, 5))
print(a)
print(a.ndim)
print(a.shape)
print(a.dtype.name)
print(type(a))<jupyter_output><empty_output><jupyter_text>Array creation There are several ways to create arrays.For example, you can create an array from a regular Python list or tuple using the array function. The type of the resulting array is deduced from the type of the elements in the sequences.<jupyter_code>a = np.array([2,3,4])
print(a)
print(a.dtype)
b = np.array([1.2, 3.5, 5.1])
print(b)
print(b.dtype)<jupyter_output><empty_output><jupyter_text>A frequent error consists in calling array with multiple numeric arguments, rather than providing a single list of numbers as an argument.<jupyter_code>np.array(1,2,3,4)    # ERROR
np.array([1,2,3,4])  # RIGHT<jupyter_output><empty_output><jupyter_text>array transforms sequences of sequences into two-dimensional arrays, sequences of sequences of sequences into three-dimensional arrays, and so on.<jupyter_code>b = np.array([(1.5,2,3), (4,5,6)])
print(b)
print(b.ndim)<jupyter_output><empty_output><jupyter_text>The function zeros creates an array full of zeros, the function ones creates an array full of ones, and the function empty creates an array whose initial content is random and depends on the state of the memory. By default, the dtype of the created array is float64.<jupyter_code>a = np.zeros((3, 4))
print(a)
print(a.ndim)
print(a.shape)
print(a.dtype.name)
a = np.ones((2,3,4), dtype=np.int16) # dtype can also be specified
print(a)
print(a.ndim)
print(a.shape)
print(a.dtype.name)<jupyter_output><empty_output><jupyter_text>To create sequences of numbers, NumPy provides a function analogous to range that returns arrays instead of lists.<jupyter_code>np.arange(10, 30, 5)<jupyter_output><empty_output><jupyter_text>It is often better to use the function linspace that receives as an argument the number of elements that we want, instead of the step<jupyter_code>np.linspace(0, 2, 9) # 9 numbers from 0 to 2<jupyter_output><empty_output><jupyter_text>Basic Operations Arithmetic operators on arrays apply *elementwise*. A new array is created and filled with the result.<jupyter_code>a = np.array([20, 30, 40, 50])
b = np.arange(4)
print('a is:', a)
print('b is:', b)

c = a-b
print('c is:', c)
b**2
10*np.sin(a)
a<35<jupyter_output><empty_output><jupyter_text>Unlike in many matrix languages, the product operator * operates elementwise in NumPy arrays. The matrix product can be performed using the @ operator or the dot function or method:<jupyter_code>A = np.array([[1, 1],
              [0, 1]])
B = np.array([[2, 0],
              [3, 4]])
A * B                        # elementwise product
A @ B                       # matrix product
A.dot(B)                    # another matrix product<jupyter_output><empty_output><jupyter_text>Some operations, such as += and *=, act in place to modify an existing array rather than create a new one.<jupyter_code>a = np.ones((2,3), dtype=int)
a
a *= 3
a
b = np.random.random((2,3))
b
b += a
b<jupyter_output><empty_output><jupyter_text>When operating with arrays of different types, the type of the resulting array corresponds to the more general or precise one (a behavior known as upcasting).<jupyter_code>a += b                  # ERROR - b is not automatically converted to integer type<jupyter_output><empty_output><jupyter_text>Many unary operations, such as computing the sum of all the elements in the array, are implemented as methods of the ndarray class.<jupyter_code>a = np.random.random((2,3))
a
a.sum()
a.min()
a.max()<jupyter_output><empty_output><jupyter_text>By default, these operations apply to the array as though it were a list of numbers, regardless of its shape. However, by specifying the axis parameter you can apply an operation along the specified axis of an array:<jupyter_code>b = np.arange(12).reshape(3,4)
b
b.sum(axis=0)                            # sum of each column
b.min(axis=1)                            # min of each row
b.cumsum(axis=1)                         # cumulative sum along each row<jupyter_output><empty_output><jupyter_text>Universal Functions NumPy provides familiar mathematical functions such as sin, cos, and exp. In NumPy, these are called “universal functions”(ufunc). Within NumPy, these functions operate elementwise on an array, producing an array as output.<jupyter_code>B = np.arange(3)
B
np.exp(B)
np.sqrt(B)
C = np.array([2., -1., 4.])
np.add(B, C)<jupyter_output><empty_output><jupyter_text>Indexing, Slicing and Iterating One-dimensional arrays can be indexed, sliced and iterated over, much like lists and other Python sequences.<jupyter_code>a = np.arange(10)**3
a
a[2]
a[2:5]
a[:6:2] = -1000    #from start to position 6, exclusive, set every 2nd element to -1000
a
a[ : :-1]                                 # reversed a
for i in a:
    print(i / 3)<jupyter_output><empty_output><jupyter_text>Multidimensional arrays can have one index per axis. These indices are given in a tuple separated by commas:<jupyter_code>def f(x,y):
    return 10*x+y

b = np.fromfunction(f,(5,4),dtype=int)
b
b[2,3]
b[0:5, 1]                       # each row in the second column of b
b[ : ,1]                        # equivalent to the previous example
b[1:3, : ]                      # each column in the second and third row of b<jupyter_output><empty_output><jupyter_text>When fewer indices are provided than the number of axes, the missing indices are considered complete slices:<jupyter_code>b[-1]                                  # the last row. Equivalent to b[-1,:]<jupyter_output><empty_output><jupyter_text>The expression within brackets in `b[i]` is treated as an i followed by as many instances of : as needed to represent the remaining axes. NumPy also allows you to write this using dots as `b[i,...]`.The dots (...) represent as many colons as needed to produce a complete indexing tuple. For example, if x is an array with 5 axes, then- `x[1,2,...]` is equivalent to `x[1,2,:,:,:]`,- `x[...,3]` to `x[:,:,:,:,3]` and- `x[4,...,5,:]` to `x[4,:,:,5,:]`.<jupyter_code>c = np.array([[[  0,  1,  2],               # a 3D array (two stacked 2D arrays)
               [ 10, 12, 13]],
              [[100,101,102],
               [110,112,113]]])
c.shape
c[1,...]                                   # same as c[1,:,:] or c[1]
c[...,2]                                   # same as c[:,:,2]<jupyter_output><empty_output><jupyter_text>Iterating over multidimensional arrays is done with respect to the first axis:<jupyter_code>b
for row in b:
    print(row)
for element in b.flat:
    print(element)<jupyter_output><empty_output><jupyter_text>Shape Manipulation  Changing the shape of an array An array has a shape given by the number of elements along each axis:<jupyter_code>a = np.floor(10*np.random.random((3,4)))
a
a.shape<jupyter_output><empty_output><jupyter_text>The shape of an array can be changed with various commands. Note that the following three commands all return a modified array, but do not change the original array:<jupyter_code>a.ravel()  # returns the array, flattened
a.reshape(6,2)  # returns the array with a modified shape
a.T  # returns the array, transposed
a.T.shape<jupyter_output><empty_output><jupyter_text>Stacking together different arrays Several arrays can be stacked together along different axes:<jupyter_code>a = np.floor(10*np.random.random((2,2)))
a
b = np.floor(10*np.random.random((2,2)))
b
np.vstack((a, b))
np.hstack((a, b))
c = np.floor(10*np.random.random((2,2)))
c
d = np.stack([a, b, c], axis=2)
d
d.shape<jupyter_output><empty_output><jupyter_text>Images are just numpy arrays Images are represented in ``scikit-image`` using standard ``numpy`` arrays.  This allows maximum inter-operability with other libraries in the scientific Python ecosystem, such as ``matplotlib`` and ``scipy``.Let's see how to build a grayscale image as a 2D array:<jupyter_code>import numpy as np
from matplotlib import pyplot as plt<jupyter_output><empty_output><jupyter_text>Make sure our plots appear inline<jupyter_code>%matplotlib inline
random_image = np.random.random([500, 500])

plt.imshow(random_image, cmap='gray')
plt.colorbar();<jupyter_output><empty_output><jupyter_text>The same holds for "real-world" images:<jupyter_code>from skimage import data

coins = data.coins()

print('Type:', type(coins))
print('dtype:', coins.dtype)
print('shape:', coins.shape)

plt.imshow(coins, cmap='gray');<jupyter_output><empty_output><jupyter_text>A color image is a 3D array, where the last dimension has size 3 and represents the red, green, and blue channels:<jupyter_code>astro = data.astronaut()

print("Shape:", astro.shape)
print("Values min/max:", astro.min(), astro.max())

plt.imshow(astro);<jupyter_output><empty_output><jupyter_text>These are *just NumPy arrays*. E.g., we can make a red square by using standard array slicing and manipulation:<jupyter_code>astro[10:110, 10:110, :] = [255, 0, 0]  # [red, green, blue]
plt.imshow(astro);<jupyter_output><empty_output><jupyter_text>Images can also include transparent regions by adding a 4th dimension, called an *alpha layer*.  Other shapes, and their meanings|Image type|Coordinates||:---|:---||2D grayscale|(row, column)||2D multichannel|(row, column, channel)||3D grayscale (or volumetric) |(plane, row, column)||3D multichannel|(plane, row, column, channel)|  Displaying images using matplotlib<jupyter_code>from skimage import data

img0 = data.chelsea()
img1 = data.rocket()
import matplotlib.pyplot as plt

f, (ax0, ax1) = plt.subplots(1, 2, figsize=(20, 10))

ax0.imshow(img0)
ax0.set_title('Cat', fontsize=18)
ax0.scatter([10, 10, 10, 10], [10, 20, 30, 40], color='white')
ax0.axis('off')

ax1.imshow(img1)
ax1.set_title('Rocket', fontsize=18)
ax1.set_xlabel(r'Launching position $\alpha=320$')

ax1.vlines([202, 300], 0, img1.shape[0], colors='magenta',
           linewidth=3, label='Side tower position')
ax1.plot([168, 190, 200], [400, 200, 300], color='white',
         linestyle='--', marker='o', label='Side angle')

ax1.legend();<jupyter_output><empty_output><jupyter_text>For more on plotting, see the [Matplotlib documentation](https://matplotlib.org/gallery/index.htmlimages-contours-and-fields) and [pyplot API](https://matplotlib.org/api/pyplot_summary.html).  Data types and image valuesIn literature, one finds different conventions for representing image values:```  0 - 255   where  0 is black, 255 is white  0 - 1     where  0 is black, 1 is white`````scikit-image`` supports both conventions--the choice is determined by thedata-type of the array.E.g., here, I generate two valid images:<jupyter_code>linear0 = np.linspace(0, 1, 2500).reshape((50, 50))
linear1 = np.linspace(0, 255, 2500).reshape((50, 50)).astype(np.uint8)

print("Linear0:", linear0.dtype, linear0.min(), linear0.max())
print("Linear1:", linear1.dtype, linear1.min(), linear1.max())

fig, (ax0, ax1) = plt.subplots(1, 2, figsize=(15, 15))
ax0.imshow(linear0, cmap='gray')
ax1.imshow(linear1, cmap='gray');<jupyter_output><empty_output><jupyter_text>The library is designed in such a way that any data-type is allowed as input,as long as the range is correct (0-1 for floating point images, 0-255 for unsigned bytes,0-65535 for unsigned 16-bit integers). You can convert images between different representations by using ``img_as_float``, ``img_as_ubyte``, etc.:<jupyter_code>from skimage import img_as_float, img_as_ubyte

image = data.chelsea()

image_ubyte = img_as_ubyte(image)
image_float = img_as_float(image)

print("type, min, max:", image_ubyte.dtype, image_ubyte.min(), image_ubyte.max())
print("type, min, max:", image_float.dtype, image_float.min(), image_float.max())
print()
print("231/255 =", 231/255.)<jupyter_output><empty_output><jupyter_text>Your code would then typically look like this:```pythondef my_function(any_image):   float_image = img_as_float(any_image)    Proceed, knowing image is in [0, 1]```We recommend using the floating point representation, given that``scikit-image`` mostly uses that format internally.  Image I/OMostly, we won't be using input images from the scikit-image example data sets.  Those images are typically stored in JPEG or PNG format.  Since scikit-image operates on NumPy arrays, *any* image reader library that provides arrays will do.  Options include imageio, matplotlib, pillow, etc.scikit-image conveniently wraps many of these in the `io` submodule, and will use whichever of the libraries mentioned above are installed:<jupyter_code>from skimage import io

image = io.imread('../data/balloon.jpg')

print(type(image))
print(image.dtype)
print(image.shape)
print(image.min(), image.max())

plt.imshow(image);<jupyter_output><empty_output><jupyter_text>We also have the ability to load multiple images, or multi-layer TIFF images.<jupyter_code>cells = io.imread('../data/cells.tif')
print('"cells" shape: {}'.format(cells.shape))
print('"cells" type: {}'.format(cells.dtype))
print('"cells" range: {}, {}'.format(cells.min(), cells.max()))<jupyter_output><empty_output><jupyter_text>We see that `cells` has 60 planes, each with 256 rows and 256 columns. We can visualize one of the 2D planes with:<jupyter_code>plt.imshow(cells[32], cmap='gray');<jupyter_output><empty_output><jupyter_text>Exercises  Exercise: visualizing RGB channelsDisplay the different color channels of the image along (each as a gray-scale image).  Start with the following template:<jupyter_code># --- read in the image ---

image = plt.imread('..data/Bells-Beach.jpg')

# --- assign each color channel to a different variable ---

r = ...
g = ...
b = ...

# --- display the image and r, g, b channels ---

f, axes = plt.subplots(1, 4, figsize=(16, 5))

for ax in axes:
    ax.axis('off')

(ax_r, ax_g, ax_b, ax_color) = axes
    
ax_r.imshow(r, cmap='gray')
ax_r.set_title('red channel')

ax_g.imshow(g, cmap='gray')
ax_g.set_title('green channel')

ax_b.imshow(b, cmap='gray')
ax_b.set_title('blue channel')

# --- Here, we stack the R, G, and B layers again
#     to form a color image ---
ax_color.imshow(np.stack([r, g, b], axis=2))
ax_color.set_title('all channels');<jupyter_output><empty_output><jupyter_text>Now, take a look at the following R, G, and B channels.  How would their combination look? (Write some code to confirm your intuition.)<jupyter_code>from skimage import draw

red = np.zeros((300, 300))
green = np.zeros((300, 300))
blue = np.zeros((300, 300))

r, c = draw.circle(100, 100, 100)
red[r, c] = 1

r, c = draw.circle(100, 200, 100)
green[r, c] = 1

r, c = draw.circle(200, 150, 100)
blue[r, c] = 1

f, axes = plt.subplots(1, 3)
for (ax, channel) in zip(axes, [red, green, blue]):
    ax.imshow(channel, cmap='gray')
    ax.axis('off')
# Hint: np.stack([...], axis=2)<jupyter_output><empty_output><jupyter_text>Exercise: Convert to grayscale ("black and white")The *relative luminance* of an image is the intensity of light coming from each point. Different colors contribute differently to the luminance: it's very hard to have a bright, pure blue, for example. So, starting from an RGB image, the luminance is given by:$$Y = 0.2126R + 0.7152G + 0.0722B$$Use Python 3.5's matrix multiplication, `@`, to convert an RGB image to a grayscale luminance image according to the formula above.Compare your results to that obtained with `skimage.color.rgb2gray`.Change the coefficients to 1/3 (i.e., take the mean of the red, green, and blue channels, to see how that approach compares with `rgb2gray`).<jupyter_code>from skimage import color, img_as_float

image = img_as_float(io.imread('../data/balloon.jpg'))

gray = color.rgb2gray(image)
my_gray = ...

# --- display the results ---

f, (ax0, ax1) = plt.subplots(1, 2, figsize=(10, 6))

ax0.imshow(gray, cmap='gray')
ax0.set_title('skimage.color.rgb2gray')

ax1.imshow(my_gray, cmap='gray')
ax1.set_title('my rgb2gray')<jupyter_output><empty_output><jupyter_text>Exercise: draw the letter HDefine a function that takes as input an RGB image and a pair of coordinates (row, column), and returns a copy with a green letter H overlaid at those coordinates. The coordinates point to the top-left corner of the H.The arms and strut of the H should have a width of 5 pixels, and the H itself should have a height of 36 pixels and width of 30 pixels.Start with the following template:<jupyter_code>def draw_H(image, coords, color=(0, 255, 0)):
    out = image.copy()
    
    ...
    
    return out<jupyter_output><empty_output><jupyter_text>Test your function like so:<jupyter_code>cat = data.chelsea()
cat_H = draw_H(cat, (50, -50))
plt.imshow(cat_H);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11917895406810226, 'fraction_numerical': 0.02704996287259998, 'mean_word_length': 4.640143583607538, 'pattern_counts': {'":': 0, '<': 154, '<?xml version=': 0, '>': 153, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '38', 'dir': 'jupyter-structured-clean-dedup', 'id': '54894', 'license': 'CC0-1.0', 'n_tokens_mistral': 6495, 'n_tokens_neox': 6075, 'n_words': 2124, 'path': 'content/lessons/00_fundamentals_skimage.ipynb', 'repo_name': 'chanzuckerberg/image-analysis-course', 'size': '18854'}