<jupyter_start><jupyter_text>Table of Contents1&nbsp;&nbsp;Case 2: Pneumonia X-ray image analysis2&nbsp;&nbsp;Background3&nbsp;&nbsp;Data4&nbsp;&nbsp;Models and training5&nbsp;&nbsp;Results and Discussion6&nbsp;&nbsp;Conclusions  Case 2: Pneumonia X-ray image analysis (Model 3)Team 16:    1. <NAME>    2. <NAME>    3. <NAME>Last modified: 27.02.2020Neural Networks for Health Technology Applications[Helsinki Metropolia University of Applied Sciences](http://www.metropolia.fi/en/)  BackgroundThe aim of this notebook is to create and train a dense neural network to classify medical images accurately with convolutional neural network using the dataset from https://data.mendeley.com/datasets/rscbjbr9sj/2   DataThe dataset used is a dataset of Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification from Mendeley Data [1].<jupyter_code># Import necessary module
%pylab inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as ts
import os

# Import libraries
from tensorflow import keras

# Import keras specific
from tensorflow.keras import layers, models, optimizers
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from sklearn.metrics import classification_report, confusion_matrix, roc_curve<jupyter_output>Populating the interactive namespace from numpy and matplotlib<jupyter_text>Load training files<jupyter_code># Training files can be found here
train_dir = "/Users/bella/Desktop/year 3/Neural Network/chest_xray/train"

# Test files can be found here
test_dir = "/Users/bella/Desktop/year 3/Neural Network/chest_xray/test"

# List training files
print('Training file directory :', os.listdir(train_dir))

# list test files
print('Testing file directory :', os.listdir(test_dir))<jupyter_output>Training file directory : ['.DS_Store', 'NORMAL', 'PNEUMONIA']
Testing file directory : ['.DS_Store', 'NORMAL', 'PNEUMONIA']<jupyter_text>Preprocess data  ScalingThe data must be preprocessed before training the network. Scale these images to a range of 0 to 1 before feeding them to the neural network model. Split dataWe will also split the training data into training and validation sets where :- Training data = 70%- Validation data = 30%<jupyter_code># Rescale training images and split training data
train_datagen = ImageDataGenerator (rescale=1./255, validation_split = 0.3)<jupyter_output><empty_output><jupyter_text>Create data generators<jupyter_code># training data generator
print('Training : ')
train_generator = train_datagen.flow_from_directory(
    train_dir, #target directory
    target_size = (224, 224), #Image resize
    batch_size = 32,
    class_mode = 'binary', # since we only have 2 classification: normal/pneumonia
    subset = "training")

# validation data generator
print('\nValidation : ')
dev_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size = (224, 224),
    batch_size = 32,
    class_mode = 'binary',
    shuffle=False,
    subset = "validation")<jupyter_output>Training : 
Found 3664 images belonging to 2 classes.

Validation : 
Found 1568 images belonging to 2 classes.

Test : 
Found 624 images belonging to 2 classes.<jupyter_text>Now we test if the train generator works :<jupyter_code># labels_batch place labels automatically. For whichever folder to come first alphabetically, it will be labelled 0. 
# e.g normal =0 and pneumonia=1
i = 0
for data_batch, labels_batch in train_generator:
#     print('data batch shape:', data_batch.shape)
#     print('labels batch shape:', labels_batch.shape)
    i = i + 1
    if i > 5:
        break  
        
# Testing continues
# Then we will check the first image from the training batch
print('\nThe first image from the training batch : ')
imshow(data_batch[0])
show()

print('If you inspect the first image from the batch, you will see that the pixel values fall in the range of 0 to 224')

# Testng continues
# We will then check the last labels batch
print('\nlabels batch : ')
labels_batch<jupyter_output>The first image from the training batch :<jupyter_text>Models and trainingWe will now create a simple CNN model. The learning network designed for the work is being referred from Chakraborty et al.(2019)[2]. The network is a CNN with 3 Convolution layers followed by 5 Dense layers including the output layer. In addition, a series of max pooling layers have been utilized in the architecture for reducing the dimensionality of the images. It is not a necessary attribute but it is an effective way to make the models efficient and increase their predictive power.<jupyter_code>model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation = 'relu', input_shape = (224, 224, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation = 'relu'))
model.add(layers.MaxPooling2D((3, 3)))
model.add(layers.Conv2D(64, (3, 3), activation = 'relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation = 'relu'))
model.add(layers.Dropout(0.5))
model.add(layers.BatchNormalization())
model.add(layers.Dense(64, activation = 'relu'))
model.add(layers.Dense(1, activation = 'sigmoid'))
model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 222, 222, 32)      896       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 111, 111, 32)      0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 109, 109, 64)      18496     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 36, 36, 64)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 34, 34, 64)        36928     
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 17, 17, 64)        0         
________________________________________________________[...]<jupyter_text>Compile the model :Before the model is ready for training, it needs a few more settings to compile.<jupyter_code>model.compile(loss = 'binary_crossentropy', 
              optimizer = 'adam', 
              metrics = ['acc'])<jupyter_output><empty_output><jupyter_text>Train the model :To start training, we call the model.fit method to "fit" the model to the data:<jupyter_code># Start the clock
start = time.time()

print('Training...', end='')
history = model.fit_generator(
    train_generator,
    steps_per_epoch = None,
    verbose = 2,
    epochs = 2,
    validation_data = dev_generator,
    validation_steps = None)

# Check the time and calculate the elapsed time and tie per epoch
end = time.time()
elapsed = end-start
time_per_epoch = elapsed/2 #number of epochs

print('Done')
print(f'Time elapsed = {elapsed:.0f} seconds')
print(f'Time per epoch = {time_per_epoch:.2f} seconds.')
# Create the graph of Training and Validation accuracy & loss vs. epochs.
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(acc))
# plot accuracy and loss lerning curves for training and validation set
plot(epochs, acc, 'bo-', label = 'Training acc')
plot(epochs, val_acc, 'r*-', label = 'Validation acc' )
title('Training and validation accuracy')
grid()
legend()
show()

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Test'], loc='upper left')
plt.show()<jupyter_output><empty_output><jupyter_text>Results and Discussion Lets use various metrics to gain insights of the model's performance. First we will have to find the best threshold from ROC curve. ROC curve<jupyter_code># Calulate the ROC curve analysis
fpr, tpr, thresholds = roc_curve(labels, predict, pos_label=1)

# Show ROC curve
plot(fpr, tpr)
plot([0,1],[0,1],'r:')
xlabel('False positive rate')
ylabel('True positive rate')
title('ROC curve')
xlim([0,1])
ylim([0,1])
grid()<jupyter_output><empty_output><jupyter_text>In ROC curve, the true positive rate (Sensitivity) is plotted in function of the false positive rate (100-Specificity) for different cut-off points. The closer the ROC curve is to the upper left corner, the higher the overall accuracy of the test (Zweig & Campbell, 1993). This means, this model has a high overall accuracy. After obtaining the TPR and FPR, we will be able to identify the best threshold. Threshold analysisThe best threshold would be obtained by using the roc curve where true positive rate(tpr) and false positive rate(fpr) overlap.<jupyter_code># Find the best threshold
plot(thresholds, 1 - fpr, label='Specificity')
plot(thresholds, tpr, label='Sensitivity')
axvline(0.93, color='red', linestyle=':')
xlim([0,1])
title('Threshold value for prediction')
xlabel('Threshold')
ylabel('Metrics value')
legend()
grid()<jupyter_output><empty_output><jupyter_text>The best threshold is 0.93 Evaluate accuracyAfter training is complete, we'll evaluate how the model performs with the best threshold.<jupyter_code># reset the validation generator and then use our trained model to make predictions on the data
print("[INFO] evaluating network...")
dev_generator.reset()
predict = model.predict_generator(dev_generator)

# for each image in the testing set we need to find the labels
labels = dev_generator.classes

# show a nicely formatted classification report
print(classification_report(labels, predict>0.93, target_names=['Normal(0)', 'Pneumonia(1)']))
# compute the confusion matrix and use it to derive the raw accuracy, sensitivity, and specificity
cm = confusion_matrix(labels, predict>0.93)
total = sum(sum(cm))
acc = (cm[0, 0] + cm[1, 1]) / total
sensitivity = cm[0, 0] / (cm[0, 0] + cm[0, 1])
specificity = cm[1, 1] / (cm[1, 0] + cm[1, 1])

# show the confusion matrix, accuracy, sensitivity, and specificity
print(cm)
print("\naccuracy: {:.4f}".format(acc))
print("sensitivity: {:.4f}".format(sensitivity))
print("specificity: {:.4f}".format(specificity))<jupyter_output>[[ 385   19]
 [  45 1119]]

accuracy: 0.9592
sensitivity: 0.9530
specificity: 0.9613
{'fraction_non_alphanumeric': 0.10596658711217184, 'fraction_numerical': 0.02711217183770883, 'mean_word_length': 5.276812462552426, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 45, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '558606', 'license': 'MIT', 'n_tokens_mistral': 3283, 'n_tokens_neox': 3020, 'n_words': 1063, 'path': 'Case 2_Model_3.ipynb', 'repo_name': 'JoeValval/neural-networks-forhealth-technology-applications', 'size': '10507'}