<jupyter_start><jupyter_text>Deep Learning  for Satellite Image Classification (Manning Publications)by <NAME>        > Chapter 2: Deliverable Solution                 Load the necessary libraries<jupyter_code>from PIL import Image, ImageDraw
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import json, os, glob
import rasterio<jupyter_output><empty_output><jupyter_text>Repurpose the ```make_plot``` function from Part 2 tutorial, using only the parts used to extract the coordinates from each lake polygon<jupyter_code>def get_data(data):
    X = []; Y = [] #pre-allocate lists to fill in a for loop
    for k in data['regions']: #cycle through each polygon
        # get the x and y points from the dictionary
        X.append(data['regions'][k]['shape_attributes']['all_points_x'])
        Y.append(data['regions'][k]['shape_attributes']['all_points_y'])
    return Y,X #image coordinates are flipped relative to json coordinates<jupyter_output><empty_output><jupyter_text>Create a new directory to store your label images<jupyter_code>os.mkdir('s2cloudless_label_imagery')
os.mkdir('s2cloudless_label_imagery'+os.sep+'data')<jupyter_output><empty_output><jupyter_text>This function gets the X,Y polygon coordinates, creates a mask, and saves the mask to file. The function is composed mostly of commands shown in the Part 2 tutorial<jupyter_code>def write_mask(data, images, all_images, i):
    X, Y = get_data(data[images[i]])
    
    # get the dimensions of the image
    nx, ny, nz = np.shape(all_images[i])
    mask = np.zeros((ny,nx))
    
    for x,y in zip(X,Y):
        # the ImageDraw.Draw().polygon function we will use to create the mask
        # requires the x's and y's are interweaved, which is what the following
        # one-liner does    
        polygon = np.vstack((x,y)).reshape((-1,),order='F').tolist()
        
        # create a mask image of the right size and infill according to the polygon
        if nx>ny:
            x,y = y,x 
            img = Image.new('L', (ny, nx), 0)
        elif ny>nx:
            x,y = y,x 
            img = Image.new('L', (ny, nx), 0)            
        else:
            img = Image.new('L', (nx, ny), 0)
        ImageDraw.Draw(img).polygon(polygon, outline=1, fill=1)
        # turn into a numpy array
        m = np.flipud(np.rot90(np.array(img)))
        try:
            mask = mask + m
        except:
            mask = mask + m.T
    matplotlib.image.imsave('s2cloudless_label_imagery'+os.sep+'data'+os.sep+images[i]+"_mask.jpg", mask.astype('uint8'))<jupyter_output><empty_output><jupyter_text>Load the VGG-JSON labels file and extract the list of sample images<jupyter_code>data = json.load(open('..'+os.sep+'2_Data'+os.sep+'s2cloudless_labels'+os.sep+'all_labels.json'))
images = sorted(data.keys())<jupyter_output><empty_output><jupyter_text>Load all the image samples into an array, like in the Part tutorial<jupyter_code>all_images = []
for image in images:
    with rasterio.open('..'+os.sep+'2_Data'+os.sep+'s2cloudless_imagery'+os.sep+'data'+os.sep+image) as dataset:
        all_images.append(dataset.read().T)<jupyter_output><empty_output><jupyter_text>Call ```write mask``` to write the image mask for each image sample, according to the information in ```data```, the json label structure<jupyter_code>for i in range(len(images)):
    write_mask(data, images, all_images, i)<jupyter_output><empty_output><jupyter_text>Use ```glob``` to load a list of these image filenames, then read then into an array ```all_labels``` one-by-one<jupyter_code>labels = sorted(glob.glob('s2cloudless_label_imagery'+os.sep+'data'+os.sep+'*.jpg'))
all_labels = []
for image in labels:
    with rasterio.open(image) as dataset:
        all_labels.append(dataset.read().T[:,:,0]) #np.flipud(np.rot90(<jupyter_output><empty_output><jupyter_text>Overlay a mask on an image to double-check that worked<jupyter_code>plt.imshow(all_images[10])
plt.imshow(all_labels[10], cmap='gray', alpha=0.5)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12596657520578697, 'fraction_numerical': 0.007233724120728361, 'mean_word_length': 4.127877237851663, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '272530', 'license': 'MIT', 'n_tokens_mistral': 1342, 'n_tokens_neox': 1319, 'n_words': 345, 'path': 'dl-satellite-image/p2_data/project_data/2. Data Acquisition and Preprocessing.ipynb', 'repo_name': 'catalin-h/liveproject', 'size': '4018'}