<jupyter_start><jupyter_text>Semantic Segmentation with KerasIn this exercise, you'll use the U-Net network to perform binary classification and segmentation for images of planes.> **Important**: Using the U-Net model is resource-intensive. before running the code in this notebook, shut down all other notebooks in this library (In each open notebook other than this one, on the **File** menu, click **Close and Halt**). If you experience and Out-of-Memory (OOM) error when running code in this notebook, shut down this entire library, and then reopen it and open only this notebook.  Install KerasTo begin with, we'll install the latest version of Keras.<jupyter_code>!pip install --upgrade keras<jupyter_output><empty_output><jupyter_text>Explore the Training DataThe training data for a U-Net model consists of two kinds of input:- **Image files**: The images that represent the *features* on which we want to train the model.- **Mask files**: Images of the object masks that the network will be trained to predict - these are the *labels*.In this example, we're going to use U-Net for binary classification of airplanes images, so there's only one class of object - and therefore one class of mask. We've deliberately made this example as simple as possible, partly to make it easier to understand what's going on, and partly to ensure it can be run in a resource-constrained environment. Let's take a look at the training images and masks:<jupyter_code>import os
from matplotlib import pyplot as plt
import skimage.io as io
import numpy as np
%matplotlib inline


fig = plt.figure(figsize=(12, 60))


train_dir = '../../data/segmentation/train'
image_dir = os.path.join(train_dir,"image/plane")
mask_dir = os.path.join(train_dir,"mask/plane")

files = os.listdir(image_dir)
rows = len(files)
cell = 0
for file in files:
    cell += 1
    
    # Open the image and mask files
    img_path = os.path.join(image_dir, file)
    img = io.imread(img_path, as_gray = True)
    
    mask_path = os.path.join(mask_dir, file)
    mask = io.imread(mask_path, as_gray = True)
    
    # plot the image
    a=fig.add_subplot(rows,3,cell)
    imgplot=plt.imshow(img, "gray")
    cell += 1

    # plot the mask
    a=fig.add_subplot(rows,3,cell)
    imgplot=plt.imshow(mask, "gray")
    cell += 1
    
    # Plot them overlaid
    a=fig.add_subplot(rows,3,cell)
    imgplot=plt.imshow(img, "gray")
    imgplot=plt.imshow(mask, "gray", alpha=0.4)

plt.show()<jupyter_output><empty_output><jupyter_text>Import the U-Net CodeThe code to implement U-Net is provided in two python files:- **model.py**: This file contains the code that implements the U-Net model- **data.py**: This file contains functions to help load and prepare training data.> **Tip**: You should explore the code in these files to get a better understanding of the way the model works.<jupyter_code>from unet_keras.data import *
from unet_keras.model import *<jupyter_output><empty_output><jupyter_text>The ouput from the code above shows the training images with their corresponding mask labels, and finally the mask overlaid on the image so you can clearly see that the masks represent the pixels that belong to the plane objects in the images.> **Note**: We deliberately chose images in which the plane objects are clearly contrasted with the background to make it easier to train with a very small number of training images and a very small amount of training!  Load the Training DataWe have a very small number of training images, so we'll apply some data augmentation to randomly flip, zoom, shear, and otherwise transform the images for each batch.<jupyter_code>data_gen_args = dict(rotation_range=0.2,
                    width_shift_range=0.05,
                    height_shift_range=0.05,
                    shear_range=0.05,
                    zoom_range=0.05,
                    horizontal_flip=True,
                    fill_mode='nearest')
train_images = trainGenerator(2,train_dir,'image','mask',data_gen_args,save_to_dir = None)<jupyter_output><empty_output><jupyter_text>Download the Model WeightsThe model has already been partially trained, so we'll download the trained weights as a starting point.<jupyter_code>!wget "https://aka.ms/unet.h5" -O ~/unet.h5<jupyter_output><empty_output><jupyter_text>Train the ModelNow we're ready to train the U-Net model. We'll train it from the training generator we created, and save the model weights after each epoch if the loss has improved. In this example, to reduce the required compute resources we'll train it for just one epoch with minimal batches. In reality, you'd need to train the model over several epochs on a GPU-based computer.> _**Note**: This will take a while on a non-GPU machine - go get some coffee!_<jupyter_code>model = unet()
home = os.path.expanduser("~")
weights_file = os.path.join(home, "unet.h5")
model.load_weights(weights_file)
model_checkpoint = ModelCheckpoint(weights_file, monitor='loss',verbose=1, save_best_only=True)
model.fit_generator(train_images,steps_per_epoch=1,epochs=1,callbacks=[model_checkpoint])<jupyter_output><empty_output><jupyter_text>Test the Trained ModelOK, let's see how well our trained model does with some images of airplanes it hasn't seen.<jupyter_code>import os
from matplotlib import pyplot as plt
import skimage.io as io
import numpy as np
%matplotlib inline

model = unet()
model.load_weights(weights_file)

fig = plt.figure(figsize=(12, 60))


test_dir = '../../data/segmentation/test'

files = os.listdir(test_dir)
rows = len(files)
cell = 0
for file in files:
    cell += 1
    # Open the file
    img_path = os.path.join(test_dir, file)
    img = io.imread(img_path, as_gray = True)
    
    src_img = img
    a=fig.add_subplot(rows,3,cell)
    imgplot=plt.imshow(src_img, "gray")
    cell += 1

    img = np.reshape(img,img.shape+(1,))
    mask_predictions = model.predict([[img]])
    mask = mask_predictions[0]
    img_mask = mask.reshape(mask.shape[0], mask.shape[1])

    a=fig.add_subplot(rows,3,cell)
    imgplot=plt.imshow(img_mask, "gray")
    cell += 1


    a=fig.add_subplot(rows,3,cell)
    imgplot=plt.imshow(src_img, "gray")
    imgplot=plt.imshow(img_mask, "binary", alpha=0.6)

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09254498714652956, 'fraction_numerical': 0.008194087403598971, 'mean_word_length': 4.297872340425532, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 33, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '489437', 'license': 'MIT', 'n_tokens_mistral': 1948, 'n_tokens_neox': 1878, 'n_words': 690, 'path': 'Mod04/02-Unet/U-Net (Keras).ipynb', 'repo_name': 'NickAjderian/dev290x', 'size': '6224'}