<jupyter_start><jupyter_text>Introducing ```numpy``` and arrays To begin processing image data, we need to understand what's going on behind the scenes.We can do that using a library called ```numpy```, which stands for __Numerical Python__. In general, you should use this library when you want to do fancy mathemtical operations with numbers, especially if you have arrays or matrices.<jupyter_code># tools for interacting with the operating system
import os

# scientific python tool for 
import numpy as np # creating an abbreviation to save keystrokes
filepath = os.path.join("..", "..", "CDS-VIS", "test_samples", "sample-data-01.csv")
np.loadtxt(fname=filepath, delimiter=',')<jupyter_output><empty_output><jupyter_text>The expression ```numpy.loadtxt(...)``` is a function call that asks Python to run the function ```loadtxt``` which belongs to the ```numpy``` library. This dotted notation is used everywhere in Python: the thing that appears before the dot contains the thing that appears after.```numpy.loadtxt``` has two parameters: the name of the file we want to read and the delimiter that separates values on a line. These both need to be character strings (or strings for short), so we put them in quotes. __Assign to variable__<jupyter_code>data = np.loadtxt(fname=filepath, delimiter=',') #comma seperated
print(data)
print(type(data))<jupyter_output><empty_output><jupyter_text>__numpy.ndarray__ tells us that we are working with an N-dimensional arrayIn this case, it's 2-dimensional<jupyter_code>print(data.dtype)
print(data.shape)
60*40<jupyter_output><empty_output><jupyter_text>__Index__ Indexing is similar to lists and strings, but we need to inlcude both row and column<jupyter_code>first_value = data[0, 0]
print(f"First value in data: {first_value}")<jupyter_output><empty_output><jupyter_text>__Question:__ What is the middle value of the array?<jupyter_code>middle_value = data[30, 20]
print(f"Middle value in data: {middle_value}")<jupyter_output><empty_output><jupyter_text>__Slice__ An index like [30, 20] selects a single element of an array, but we can select whole sections as well. For example, we can select the first ten columns of values for the first four rows like this:<jupyter_code>print(data[0:4, 0:10])<jupyter_output><empty_output><jupyter_text>First ten columns, rows five-ten<jupyter_code>print(data[5:10, 0:10])<jupyter_output><empty_output><jupyter_text>__Select only one row__<jupyter_code>data[0,:]<jupyter_output><empty_output><jupyter_text>__Select only one column__<jupyter_code>data[:,0]<jupyter_output><empty_output><jupyter_text>__Numpy functions__<jupyter_code>np.mean(data)
max_value, min_value, std_dev = np.max(data), np.min(data), np.std(data)
print(f"Maximum: {max_value}")
print(f"Minimum: {min_value}")
print(f"Standard deviation: {std_dev}")<jupyter_output><empty_output><jupyter_text>Show numpy + dot + tab, access full range of options. Show ```help()```<jupyter_code>help(np.count_nonzero)<jupyter_output><empty_output><jupyter_text>__Operation across rows__<jupyter_code>print(np.mean(data, axis=0))
print(np.mean(data, axis=0).shape)<jupyter_output><empty_output><jupyter_text>"Average score per day" __Operation along columns__ "Average score per patient"<jupyter_code>print(np.mean(data, axis=1))
print(np.mean(data, axis=1).shape)<jupyter_output><empty_output><jupyter_text>This is a good overview to show how things work wiht ```numpy```:https://www.sharpsightlabs.com/blog/numpy-axes-explained/  Basic image processing with OpenCV We start by loading all of the modules we'll need for this class<jupyter_code># We need to incldue the home directory in our path, so we can read in our own module.
import sys
sys.path.append(os.path.join("..", "..", "CDS-VIS"))
# python framework for working with images
import cv2

# some utility functions for plotting images
from utils.imutils import jimshow<jupyter_output><empty_output><jupyter_text>__Read image__ We can load an image using a handy function from OpenCV<jupyter_code>path_to_image = os.path.join("..", "..", "CDS-VIS", "img", "trex.png")
image = cv2.imread(path_to_image)
jimshow(image, "Image")<jupyter_output><empty_output><jupyter_text>__Save image__<jupyter_code>filepath, _ = os.path.split(path_to_image)
outfile = os.path.join(filepath, "new_dino.jpg")
cv2.imwrite(outfile, image)<jupyter_output><empty_output><jupyter_text>__Inspect image__<jupyter_code>print(type(image))
print(image.shape)<jupyter_output><empty_output><jupyter_text>What is an image? __Remember how ```numpy``` arrays work!__ROWSxCOLUMNS == HEIGHTxWIDTH<jupyter_code>height = image.shape[0]
width = image.shape[1]
print(f"[INFO] Image height: {height} pixels")
print(f"[INFO] Image width: {width} pixels")<jupyter_output><empty_output><jupyter_text>In our image, there are 228*350 = 79,800 pixels __What about the last one?__<jupyter_code>image.shape[2]<jupyter_output><empty_output><jupyter_text>__NB!__```OpenCV``` stores RGB tuples in REVERSE ORDER __What colour is a specific pixel?__<jupyter_code>b, g, r = image[0, 0]
print(f"[INFO] pixels at (0, 0) - Red: {r}, Green {g}, Blue: {b}")<jupyter_output><empty_output><jupyter_text>__Modify colour__<jupyter_code>image[0, 0] = (0, 0, 255) # remember - blue, green, red!
(b, g, r) = image[0, 0]
print(f"[INFO] pixels at (0, 0) - Red: {r}, Green {g}, Blue: {b}")<jupyter_output><empty_output><jupyter_text>__Image slice__<jupyter_code>corner = image[0:100, 0:100]
jimshow(corner, "Corner")<jupyter_output><empty_output><jupyter_text>__Change corner colour__<jupyter_code>image[0:100,0:100] = (0, 0, 255)
jimshow(image, "Update")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12582310019576437, 'fraction_numerical': 0.015483182060864922, 'mean_word_length': 7.822605965463108, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 93, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '169937', 'license': 'MIT', 'n_tokens_mistral': 2031, 'n_tokens_neox': 1949, 'n_words': 516, 'path': 'notebooks/session1_inclass_rdkm.ipynb', 'repo_name': 'sarah-hvid/cds-visual', 'size': '5619'}