<jupyter_start><jupyter_text>Exercise 2: Data set, augmentation, and baselineIn this exercise we are going to explore how augmented data can be used in training and testing.The code used in the exercise is based on the code already shown during the lecture. In this notebook you get the main code blocks to do the exercise. It helps to look at the code to get inspiration to solve the exercise.<jupyter_code>import numpy as np
import pandas as dp
import skimage.io as io
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Load dataIn this execercise we will be working with a CT data set from lungs which are acompanied by a list of labels to tell if the image is healthy or malign. The image slices are stored in a (3D) numpy array and the labels in a pandas data frame. [Pandas](https://pandas.pydata.org/docs/) is a libray that helps you to store tables of data that are searchable and slicable depending on different criteria.__Note:__ Please uncompress the file ```ct_tiles.tif.zip``` before starting.<jupyter_code>img = io.imread('ct_tiles.tif')
labels = dp.read_csv('malignancy.csv')
print('Image size: {0},{1},{2}'.format(img.shape[0],img.shape[1],img.shape[2]))
print('Labels {0}'.format(labels.malignancy.count() ))<jupyter_output>Image size: 6691,64,64
Labels 6691<jupyter_text>Look at the first five labels<jupyter_code>labels.head()<jupyter_output><empty_output><jupyter_text>Look at the first five slices<jupyter_code>plt.figure(figsize=[15,5])
for idx,tile in enumerate(img[0:5]) :
    plt.subplot(1,5,idx+1)
    plt.imshow(tile) ;
    plt.title('Malignancy = {0}'.format(labels.malignancy[idx]))<jupyter_output><empty_output><jupyter_text>Look at the two casesIn our exercise we will work with images of healthy (imgA) and malign sample (imgB).<jupyter_code>imgA=img[labels[labels.malignancy==0.0].idx.iloc[3]]
imgB=img[labels[labels.malignancy==1.0].idx.iloc[3]]
plt.figure(figsize=[12,7])
plt.subplot(1,2,1)
plt.imshow(imgA)
plt.title('Healthy')

plt.subplot(1,2,2)
plt.imshow(imgB)
plt.title('Malign');<jupyter_output><empty_output><jupyter_text>Augmenting the dataLet's create a data augmentator. Here, the values from the lecture are used, but you can play with different settings for parameters like:- rotation- translation- scaling- etc.<jupyter_code>from keras.preprocessing.image import ImageDataGenerator
img_aug = ImageDataGenerator(
    featurewise_center=False,
    samplewise_center=False,
    zca_whitening=False,
    zca_epsilon=1e-06,
    rotation_range=30.0,
    width_shift_range=0.25,
    height_shift_range=0.25,
    shear_range=0.25,
    zoom_range=0.5,
    fill_mode='nearest',
    horizontal_flip=False,
    vertical_flip=False
)<jupyter_output><empty_output><jupyter_text>This is an example using the lecture code with the lung data.<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
%matplotlib inline

tiles = np.expand_dims(img[0:6500:450], -1)
tileLabels = labels.malignancy[0:6500:450]
fig, m_axs = plt.subplots(4, 10, figsize=(14, 9))
# setup augmentation
img_aug.fit(tiles)
real_aug = img_aug.flow(tiles[:10], tileLabels[:10], shuffle=False)

aug_tiles  = tiles
aug_labels = tileLabels

for c_axs, do_augmentation in zip(m_axs, [False,True, True, True, True,True,True,True,True,True]):
    if do_augmentation:
        img_batch, label_batch = next(real_aug)
        aug_tiles=np.concatenate((aug_tiles,img_batch))
        aug_labels=np.concatenate((aug_labels,label_batch))
    else:
        img_batch, label_batch = tiles, tileLabels
    for c_ax, c_img, c_label in zip(c_axs, img_batch, label_batch):
        c_ax.imshow(c_img[:, :, 0], cmap='viridis', vmin=0, vmax=255)
        c_ax.set_title('{}\n{}'.format(
            c_label, 'aug' if do_augmentation else ''))
        c_ax.axis('off')
aug_tiles.shape
aug_labels.shape<jupyter_output><empty_output><jupyter_text>Testing baselines with augmented dataFrom the lectures you learned that in order to compare the performance of an algorithm, you need a baseline algorithm. Previously, you heard about two algorithms:- Dummy classifier which uses the majority as prediction for any input data.- Nearest neighbor that uses the nearest feature as prediction.Below you get examples on the lung data.  Dummy classifier<jupyter_code>from sklearn.dummy import DummyClassifier
dc = DummyClassifier(strategy='most_frequent')
dc.fit(tiles, tileLabels)
dc.predict(img[1:6000:1000])<jupyter_output><empty_output><jupyter_text>Nearest neighbor<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
neigh_class = KNeighborsClassifier(n_neighbors=1)
neigh_class.fit(tiles.reshape((15, -1)), tileLabels)
# predict on a few images
neigh_class.predict(tiles.reshape((15, -1)))<jupyter_output><empty_output><jupyter_text>Confusion matrix<jupyter_code>import seaborn as sns
import pandas as pd
from sklearn.metrics import accuracy_score, confusion_matrix
def print_confusion_matrix(confusion_matrix, class_names, figsize = (10,7), fontsize=14):
    """Prints a confusion matrix, as returned by sklearn.metrics.confusion_matrix, as a heatmap.
    
    Stolen from: https://gist.github.com/shaypal5/94c53d765083101efc0240d776a23823
    
    Arguments
    ---------
    confusion_matrix: numpy.ndarray
        The numpy.ndarray object returned from a call to sklearn.metrics.confusion_matrix. 
        Similarly constructed ndarrays can also be used.
    class_names: list
        An ordered list of class names, in the order they index the given confusion matrix.
    figsize: tuple
        A 2-long tuple, the first value determining the horizontal size of the ouputted figure,
        the second determining the vertical size. Defaults to (10,7).
    fontsize: int
        Font size for axes labels. Defaults to 14.
        
    Returns
    -------
    matplotlib.figure.Figure
        The resulting confusion matrix figure
    """
    df_cm = pd.DataFrame(
        confusion_matrix, index=class_names, columns=class_names, 
    )
    fig, ax1 = plt.subplots(1, 1, figsize=figsize)
    try:
        heatmap = sns.heatmap(df_cm, annot=True, fmt="d")
    except ValueError:
        raise ValueError("Confusion matrix values must be integers.")
    heatmap.yaxis.set_ticklabels(heatmap.yaxis.get_ticklabels(), rotation=0, ha='right', fontsize=fontsize)
    heatmap.xaxis.set_ticklabels(heatmap.xaxis.get_ticklabels(), rotation=45, ha='right', fontsize=fontsize)
    plt.ylabel('True label')
    plt.xlabel('Predicted label')
    return ax1
pred_values = neigh_class.predict(tiles.reshape((15, -1)))
ax1 = print_confusion_matrix(confusion_matrix(tileLabels, pred_values), class_names=range(2))
ax1.set_title('Accuracy: {:2.2%}'.format(accuracy_score(tileLabels, pred_values)));<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10391722099039172, 'fraction_numerical': 0.02394678492239468, 'mean_word_length': 5.057296329453894, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '597588', 'license': 'Apache-2.0', 'n_tokens_mistral': 2334, 'n_tokens_neox': 2222, 'n_words': 592, 'path': 'Exercises/02-Augmentation/02-AugmentationExercise.ipynb', 'repo_name': 'kimdon-ethz/Quantitative-Big-Imaging-2022', 'size': '6765'}