<jupyter_start><jupyter_text>**Megha**A CNN for ground based cloud classification and weather forecating.***By : [<NAME>](https://www.github.com/Naimish240)***  **Table Of Contents**1. Download the Dataset into Colab2. Pre Processing3. Building the Model4. Training the Model5. Testing the Model6. Exporting the Model to tflite **Downloading the Dataset**The dataset was procured from [Harvard DataVerse](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/CADDPD) and uploaded to [my Google Drive](https://drive.google.com/file/d/1oC73P4BAl2fkcu4tETL7NpQlKWlvZe8O/view?usp=sharing) for easy access in colab.<jupyter_code># Download the Dataset
!gdown --id 1oC73P4BAl2fkcu4tETL7NpQlKWlvZe8O
# Extract the Images
!tar -xzvf  'CloudDataset.tar.gz'
# Delete the Tarfile
!rm 'CloudDataset.tar.gz'<jupyter_output>[1;30;43mStreaming output truncated to the last 5000 lines.[0m
Cloud Images/._As-N046.jpg
Cloud Images/As-N046.jpg
Cloud Images/._Sc-N166.jpg
Cloud Images/Sc-N166.jpg
Cloud Images/._Cs-N159.jpg
Cloud Images/Cs-N159.jpg
Cloud Images/._Sc-N172.jpg
Cloud Images/Sc-N172.jpg
Cloud Images/._As-N052.jpg
Cloud Images/As-N052.jpg
Cloud Images/._St-N126.jpg
Cloud Images/St-N126.jpg
Cloud Images/._Cs-N165.jpg
Cloud Images/Cs-N165.jpg
Cloud Images/._Ns-N199.jpg
Cloud Images/Ns-N199.jpg
Cloud Images/._Cs-N171.jpg
Cloud Images/Cs-N171.jpg
Cloud Images/._St-N132.jpg
Cloud Images/St-N132.jpg
Cloud Images/._Cc-N197.jpg
Cloud Images/Cc-N197.jpg
Cloud Images/._Ci-N037.jpg
Cloud Images/Ci-N037.jpg
Cloud Images/._Ac-N088.jpg
Cloud Images/Ac-N088.jpg
Cloud Images/._Cb-N183.jpg
Cloud Images/Cb-N183.jpg
Cloud Images/._Cb-N197.jpg
Cloud Images/Cb-N197.jpg
Cloud Images/._Cc-N183.jpg
Cloud Images/Cc-N183.jpg
Cloud Images/._Ci-N023.jpg
Cloud Images/Ci-N023.jpg
Cloud Images/._Cu-N100.jpg
Cloud Images/Cu-N100.jpg[...]<jupyter_text>Check if images downloaded properly<jupyter_code>import glob
!ls<jupyter_output>'Cloud Images'	 sample_data<jupyter_text>Get list of all images into memory.<jupyter_code>IMAGE_PATHS = glob.glob('Cloud Images/*.jpg')

# Print statements to check if it's working properly
print(len(IMAGE_PATHS))
IMAGE_PATHS[:5]
IMAGE_PATHS[0][13:15]<jupyter_output><empty_output><jupyter_text>**Pre Processing**<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import PIL
from skimage.transform import resize
import random

random.seed(69)

# Shuffle images here itself, for sanity reasons
random.shuffle(IMAGE_PATHS)<jupyter_output><empty_output><jupyter_text>Load a random image and display it<jupyter_code>img_path = random.choice(IMAGE_PATHS)
print(img_path)

img = plt.imread(img_path)
plt.imshow(img)
def img_to_pil(img_path):
  '''
    Input:
      img_path : path to image on disk

    Output:
      img : PIL image object
  '''
  img = PIL.Image.open(img_path)
  return img

def img_to_numpy_array(img):
  '''
  Input:
    img : PIL object

  Output:
    Numpy Array of image
  '''
  return np.array(img)
# Gets image to numpy
img = img_to_pil(img_path)
img = img_to_numpy_array(img)
# Shows image
plt.imshow(img)<jupyter_output><empty_output><jupyter_text>**Get Labels from images**<jupyter_code>CATEGORIES = ["Ci", "Cs", "Cc", "Ac", "As", "Cu", "Cb", "Ns", "Sc", "St", "Ct"]

y = []

for image in IMAGE_PATHS:
  image_category = image[13:15]
  y.append(CATEGORIES.index(image_category))

# Check if it worked properly
print(IMAGE_PATHS[0])
print(y[0])<jupyter_output>Cloud Images/Sc-N005.jpg
8<jupyter_text>*** **Augment Images**Currently, the dataset contains :- Cirrus        (139 images)- Cirrostratus  (287 images)- Cirrocumulus  (268 images)- Altocumulus   (221 images)- Altostratus   (188 images)- Cumulus       (182 images)- Cumulonimbus  (242 images)- Nimbostratus  (274 images)- Stratocumulus (340 images)- Stratus       (202 images)- Contrails     (200 images)Total :         2543 imagesTo balance the dataset, we'll assume that we need 350 images in each class (since max + 10)So, extending this, each class requires- Cirrus        (211 images)- Cirrostratus  (063 images)- Cirrocumulus  (082 images)- Altocumulus   (129 images)- Altostratus   (162 images)- Cumulus       (168 images)- Cumulonimbus  (108 images)- Nimbostratus  (076 images)- Stratocumulus (010 images)- Stratus       (148 images)- Contrails     (150 images)Total :         3850 images**Note**:*Since all we're doing is looking for patterns and not features, we're assuming that augmenting a class tomore than double the images it initially started off with won't have a serious impact on the final results.This may/may not be a valid assumption, though. We'll only know for sure after training.Also, since all we're looking for is a pattern, rotation and flipping don't make much sense. Same goes for adding noise and blurring. And given how some images have buildings in the bottom, translating is also risky.Hence, we're only cropping or resizing the images.****<jupyter_code># Function to crop image array
def cropImg(x, y, arr):
  return arr[x:x+NEW_IMAGE_SIZE, y:y+NEW_IMAGE_SIZE]

# Function to augment images and get feature set
def augmented_images():
  # Features set
  features = []
  # Labels set
  labels = []

  # Iterate through all images in the dataset to make the train set
  for i in range(len(IMAGE_PATHS)):

    # If we don't need to augment any more images of that class
    if not augmentable_count[y[i]] > 0:
      continue

    # Reduce augmentable image count by 1
    augmentable_count[y[i]] -= 1

    # Get Numpy array of image in grayscale mode
    img_arr = img_to_numpy_array(img_to_pil(IMAGE_PATHS[i]))
    
    '''
    if img_arr.shape == (256, 256, 3):
      
      continue
    # print(img_arr.shape)
    '''
    OLD_IMAGE_SIZE = img_arr.shape[0]

    # Get coordinates to crop into
    x_coordinate = random.randint(0, OLD_IMAGE_SIZE - NEW_IMAGE_SIZE - 1)
    y_coordinate = random.randint(0, OLD_IMAGE_SIZE - NEW_IMAGE_SIZE - 1)

    # Crop image
    new_img = cropImg(x_coordinate, y_coordinate, img_arr)
    #print(new_img.shape)

    # Add it to the features set
    features.append(img_to_numpy_array(new_img))

    labels.append(y[i])

  # Return Features set
  return features, labels

def resized_images():
  features = []
  labels = []

  for i in range(len(IMAGE_PATHS)):
    # Get image array
    img_arr = img_to_numpy_array(img_to_pil(IMAGE_PATHS[i]))
    #print(img_arr.shape)

    '''
    if img_arr.shape == (256, 256, 3):
      features.append(img_arr)
      labels.append(y[i])
      continue
    '''

    res = resize(img_arr, (128, 128, 3))
    #print(res.shape)
    
    features.append(res)
    #print(i)
    labels.append(y[i])

  return features, labels

# Set image dimensions
NEW_IMAGE_SIZE = 128

# Count of number of images that can be augmented per class
augmentable_count = {
    0  : 211,
    1  :  63,
    2  :  82,
    3  : 129,
    4  : 162,
    5  : 168,
    6  : 108,
    7  :  76,
    8  :  10,
    9  : 148,
    10 : 150
}<jupyter_output><empty_output><jupyter_text>**Crop Images**<jupyter_code>x_aug, y_aug = augmented_images()
print(len(x_aug), len(y_aug))
print(x_aug[0].shape)

img = random.choice(x_aug)
plt.imshow(img)<jupyter_output>1235 1235
(128, 128, 3)<jupyter_text>**Resize Images**<jupyter_code>x_res, y_res = resized_images()
print(len(x_res), len(y_res))
print(x_res[0].shape)

img = random.choice(x_res)
plt.imshow(img)<jupyter_output>2543 2543
(128, 128, 3)<jupyter_text>**Merge into single dataset**<jupyter_code>x = x_res + x_aug
y = y_res + y_aug

print(x[2556].shape)

x = np.array(x)
print(x.shape)

y = np.stack(y, axis=0)
print(y.shape)

print(len(x))
print(len(y))

x /= 255
# Delete old stuff to prevent mem leaks
del x_res
del x_aug
del y_res
del y_aug
del img
del IMAGE_PATHS<jupyter_output><empty_output><jupyter_text>As you can see, we wanted 3850 images, but augmentation only got us 3778 images. Which is good enough for the classifier.Before we move ahead, let's randomly plot a few images to see the results of the augmentation.<jupyter_code># Makes sense, since scaled down by 255
img = x[0]
print(x[0].shape)
plt.imshow(img)<jupyter_output>(128, 128, 3)<jupyter_text>**Build the Model**<jupyter_code>import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers<jupyter_output><empty_output><jupyter_text>**Test Train Split**<jupyter_code>x_train = x[:-250]
x_test = x[-250:]
y_train = y[:-250]
y_test = y[-250:]

# Print shapes to ensure things are as expected
print(x_train.shape)
print(x_test.shape)
print(y_train.shape)
print(y_test.shape)

# Delete to prevent memory leaks
del x
del y<jupyter_output>(3528, 128, 128, 3)
(250, 128, 128, 3)
(3528,)
(250,)<jupyter_text>**Build the Model**<jupyter_code># Create the Model
num_classes = 11

model = keras.Sequential()

model.add(keras.Input(shape=(128, 128, 3)))

# Add First Conv Layer
model.add(layers.Conv2D(
    filters=64, 
    kernel_size=5, 
    strides=(3,3), 
    activation='relu',
))


# Do some BatchNorm
model.add(keras.layers.BatchNormalization())
# Throw in a Max Pooling layer
model.add(layers.MaxPooling2D(
    pool_size=(3,3)
))

# Add second Conv Layer
model.add(layers.Conv2D(
    filters=224, 
    kernel_size=(3,3), 
    strides=(1,1), 
    activation='relu', 
    padding="same"
))
          
# Do some BatchNorm
model.add(keras.layers.BatchNormalization())

# Add another Conv2D layer
model.add(layers.Conv2D(
    filters=96,
    kernel_size=(3,3),
    strides=(1,1),
    activation='relu',
    padding='same'
))

# Add another Conv2D layer
model.add(layers.Conv2D(
    filters=32,
    kernel_size=(3,3),
    strides=(1,1),
    activation='relu',
    padding='same'
))

# MaxPool layer
model.add(layers.MaxPooling2D(
    pool_size=(1,1)
))

# Time to go Flat
model.add(layers.Flatten())

# Add a Dense layer
model.add(layers.Dense(320, 'relu'))
# One Final Dropout
model.add(layers.Dropout(0.5))
# Output Layer
model.add(layers.Dense(11, 'softmax'))

# Compile the model
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

# Build model
model.build()

# Model Summary
model.summary()

# Train
epochs = 45
history = model.fit(
    x_train, 
    y_train, 
    batch_size=32,
    epochs=epochs
)
# Analysis, of first training part
acc = history.history['accuracy']

loss = history.history['loss']

epochs_range = range(epochs)

plt.figure(figsize=(8, 8))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.legend(loc='lower right')
plt.title('Training Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.legend(loc='upper right')
plt.title('Training Loss')
plt.show()
# Train
epochs = 10
history = model.fit(
    x_train, 
    y_train, 
    batch_size=32,
    epochs=epochs
)
# Analysis
acc = history.history['accuracy']

loss = history.history['loss']

epochs_range = range(epochs)

plt.figure(figsize=(8, 8))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.legend(loc='lower right')
plt.title('Training Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.legend(loc='upper right')
plt.title('Training Loss')
plt.show()<jupyter_output><empty_output><jupyter_text>Yeah, since the model didn't improve much during these 10 epochs, there's a very high probability that the model will start overfitting after this. So, we quit while we're ahead and move on to exporting this model.<jupyter_code>model.save('.')
converter = tf.lite.TFLiteConverter.from_saved_model('.')
tflite_model = converter.convert()

# Save the model.
with open('model.tflite', 'wb') as f:
  f.write(tflite_model)
!ls
from google.colab import files
files.download("model.tflite")
files.download("saved_model.pb")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12360027146250424, 'fraction_numerical': 0.042161520190023755, 'mean_word_length': 4.39295516925892, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 56, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '559753', 'license': 'Apache-2.0', 'n_tokens_mistral': 4723, 'n_tokens_neox': 4261, 'n_words': 1084, 'path': 'Megha.ipynb', 'repo_name': 'MeghaApps/tf-model', 'size': '11796'}