<jupyter_start><jupyter_text>What is an image ?An image is a 2D data with one or more channels for each pixels: grayscale image have 1 channels, RGB images have 3 channels, hyperspectrale images might have dozen of channels.The best way to process images is to use an existing image library. There are multiple image libraries based on the process' needs. For this notebook, the Scikit-image *aka* skimage is used. Namely, it is a bad idea to use matplotlib to read and display an image. Import an image<jupyter_code># Get the path to the Pikachu image and load image in img variable
path_pikachu: Path = Path.cwd() / "images" / "pikachu.png"
img = io.imread(path_pikachu)

# Display the image
# In future this code is inside the function iprint in the repo package NLI
# https://github.com/AwePhD/NotebooksLabsessionImage/blob/main/NLI/utils.py#L6
# https://github.com/AwePhD/NotebooksLabsessionImage/blob/main/NLI/utils.py#L20
# Display
plt.figure()
plt.axis("off")
plt.title("The best rat ever")
io.imshow(img)
plt.show()<jupyter_output><empty_output><jupyter_text>Use of skimage modules to change range and data type of an image.<jupyter_code># Base image
print(
    f"type(img): {type(img)}\n"
    f"img.shape: {img.shape}\n"
    f"img.dtype: {img.dtype}\n"
    f"Range: {img.min()}-{img.max()}\n"
)
# To float
img_float = img_as_float(img)
print_img_info(img_float)

# To uint8 (unsigned byte)
img_ubyte = img_as_ubyte(img)
print_img_info(img_ubyte)

# Note that the base image was already on 8 bits (1 Byte) unsigned.<jupyter_output>type(img): <class 'numpy.ndarray'>
img.shape: (120, 120, 4)
img.dtype: uint8
Range: 0-255

img.shape: (120, 120, 4)
img.dtype: float64
Range of value: 0.0-1.0

img.shape: (120, 120, 4)
img.dtype: uint8
Range of value: 0-255<jupyter_text>Use of numpy witn ndarray for Image processing Smart artihmetic operation on ndarray<jupyter_code># Numpy is smart
# Increment all value by a scalar
data: np.ndarray = np.random.randn(50,50,300)
data_plus_one: np.ndarray = data + 1
print(
    f"data[25,30,120]: {data[25,30,120]:.2f}\n"
    f"data_plus_one[25,30,120]: {data_plus_one[25,30,120]:.2f}\n"
)

# Increment all values by a vector
# All element with the same last indice get the same increment
data_plus_vector: np.ndarray = data + np.arange(300)
print(
    f"np.arange(300)[120]: {np.arange(300)[120]}\n"
    f"{'':-^50}\n"
    f"data[25,30,120]: {data[25,30,120]:.2f}\n"
    f"data_plus_vector[25,30,120]: {data_plus_vector[25,30,120]:.2f}\n"
    f"{'':-^50}\n"
    f"data[24,0,120]: {data[24,0,120]:.2f}\n"
    f"data_plus_vector[24,0,120]: {data_plus_vector[24,0,120]:.2f}\n"
    f"{'':-^50}\n"
    f"data[25,30,121]: {data[25,30,121]:.2f}\n"
    f"data_plus_vector[25,30,121]: {data_plus_vector[25,30,121]:.2f}\n"
)
# See that coefficient with last slice 120 got incremented by 120 and a
# different coefficient with last slice 121 got incremented by 121.
# Numpy smartly increments data
# Process mean

# Let's have a 2D array with 50 elements of 100 length
# A 2D matrix represents 50 vectors having 100 coordinates.
ar: np.ndarray = np.random.randn(50,100)

# We would like to have the mean value of the array
# Namely the mean value given any vector and any coordinate
print(f"ar.mean(): {ar.mean():.3f}\n")

# We would like to have the mean value for each coordinate
# That's the mean along the first axis, the axis of vector
# The mean of first coordinate is the mean of all first coordinates for all
# vectors.
# ar[:,0] is the first coordinate for all vector
mean_coord: np.ndarray = ar.mean(0)
print(f"ar.mean(0).shape: {mean_coord.shape}\n")

# We would like to have the mean value for each vector
# That's the mean along the second exis, the axis of coordinates
# The mean of first vector is the mean of its coordinates 
# ar[0,:] (or ar[0]) is the first vector
mean_vector: np.ndarray = ar.mean(1)
print(f"ar.mean(0).shape: {mean_vector.shape}\n")<jupyter_output>ar.mean(): 0.008

ar.mean(0).shape: (100,)

ar.mean(0).shape: (50,)<jupyter_text>Compute the RGB mean of an image on list comprehensionHow to get data from files and path is explained in the Path collab.[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AwePhD/NotebooksLabsessionImage/blob/main/notebooks/manage_path.ipynb) List comprehensionA list comprehension is faster than a regular loop ([SO reply](https://stackoverflow.com/a/22108640)) and more natural than a loop structure - once we are used to it! List comprehension are **only** meant to build a list from another object.<jupyter_code># Let's have a list of name
name_list: List[str] = ["Alice", "Adam", "Bob", "Bertrand", "Céline"]
print(f"name_list: {name_list}\n")

# We want to build the list of names with 'A' as first letter
# The classical approach is
A_name_list: List[str] = []
for name in name_list:
  if name[0] == 'A':
    A_name_list.append(name)
print(f"{' No list comprehension ':-^50}\n")
print(f"A_name_list: {A_name_list}\n")

# The list comprenhension approach is
A_name_list_c: List[str] = [
  name
  for name in name_list
  if name[0] == 'A'
]
print(f"{' List comprehension ':-^50}\n")
print(f"A_name_list: {A_name_list_c}\n")<jupyter_output>name_list: ['Alice', 'Adam', 'Bob', 'Bertrand', 'Céline']

------------- No list comprehension --------------

A_name_list: ['Alice', 'Adam']

--------------- List comprehension ---------------

A_name_list: ['Alice', 'Adam']<jupyter_text>Computing RGB mean of pictures on list comprehension<jupyter_code>path_images_dir: Path = Path.cwd() / "images"
mean_list: List[np.ndarray] = [
  img_as_float(
      io.imread(path_file)
  ).mean((0,1))
  for path_file in path_images_dir.iterdir()
]

# Print few elements
print(
    f"{'3 first elements of the list of means':^40}"
)
pprint(mean_list[:3])

# Note, the mean vectors are lame and not very representative in this case.
# A lot of pixel are taken in consideration while their pixel have are white 
# or / and transparent The solution would be to compute the mean from a masked 
# image version. 

# This function computes the problem correctly for .png image.
# https://github.com/AwePhD/NotebooksLabsessionImage/blob/main/NLI/utils.py#L42<jupyter_output>3 first elements of the list of means  
[array([0.11370833, 0.05174619, 0.0409055 , 0.12972685]),
 array([0.04815142, 0.0482658 , 0.04826498, 0.06252315]),
 array([0.20425218, 0.18477397, 0.16943927, 0.32776389])]<jupyter_text>Dictionnary comprehension<jupyter_code># Let's have a dictionnary for each member of a class group and their grades.
# A dictionnary is similar of arrays: element of a dictionnary is indexed by an
# object (str, int, float) while array's element are indexed by a sequence of 
# integer.
grad_dict: Dict[str, int] = {
    "Alice": 20,
    "Adam": 8,
    "Bob": 6,
    "Bertrand": 20,
    "Céline": 15,
    "Sarah": 19
}
pprint(grad_dict)
print(f"grad_dict['Céline']: {grad_dict['Céline']}\n")

# We want the students with grades better than 10
# Classical way
better_than_average: Dict[str, int] = {}
for k,v in grad_dict.items():
  if v >= 10:
    better_than_average[k] = v
print(f"{' No dict comprehension ':-^80}\n")
print(f"better_than_average: {better_than_average}\n")

# Dict comprehension way
better_than_average_c: Dict[str, int] = {
    k: v
    for k,v in grad_dict.items()
    if v >= 10
}
print(f"{' Dict comprehension ':-^80}\n")
print(f"better_than_average_c: {better_than_average_c}\n")<jupyter_output>{'Adam': 8, 'Alice': 20, 'Bertrand': 20, 'Bob': 6, 'Céline': 15, 'Sarah': 19}
grad_dict['Céline']: 15

---------------------------- No dict comprehension -----------------------------

better_than_average: {'Alice': 20, 'Bertrand': 20, 'Céline': 15, 'Sarah': 19}

------------------------------ Dict comprehension ------------------------------

better_than_average_c: {'Alice': 20, 'Bertrand': 20, 'Céline': 15, 'Sarah': 19}<jupyter_text>Computing RGB mean of pictures on list comprehension<jupyter_code># Dictionnary is a special variable that as a "key" to index the "values"
# An array, a tuple, a list have an integer to index each value for example.
path_images_dir: Path = Path.cwd() / "images"
mean_dict: Dict[Path, np.ndarray] = {
  path_file: img_as_float(
      io.imread(path_file)
  ).mean((0,1))
  for path_file in path_images_dir.iterdir()
}

# Print few elements
print(
    f"||{'3 first elements of the list of means':^55}||"
)
pprint(list(mean_dict.items())[:3])

# Mapping the path of a picture and its mean might be convenient for future
# uses ...<jupyter_output>||         3 first elements of the list of means         ||
[(PosixPath('/content/images/octillery.png'),
  array([0.11370833, 0.05174619, 0.0409055 , 0.12972685])),
 (PosixPath('/content/images/nincada.png'),
  array([0.04815142, 0.0482658 , 0.04826498, 0.06252315])),
 (PosixPath('/content/images/terrakion.png'),
  array([0.20425218, 0.18477397, 0.16943927, 0.32776389]))]
{'fraction_non_alphanumeric': 0.1532167675409101, 'fraction_numerical': 0.05929163864604349, 'mean_word_length': 4.2736406619385345, 'pattern_counts': {'":': 6, '<': 24, '<?xml version=': 0, '>': 26, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '96504', 'license': 'MIT', 'n_tokens_mistral': 3596, 'n_tokens_neox': 3186, 'n_words': 958, 'path': 'notebooks/image_processing.ipynb', 'repo_name': 'AwePhD/Notebooks_Labsession_Image', 'size': '8922'}