<jupyter_start><jupyter_text>Assignment-2 : Tomato Leaf Disease Prediction  _(by Transfer Learning using VGG-16 Network)_  Initial Set-Up<jupyter_code># information about the 
!nvidia-smi
!pip install tensorflow-gpu<jupyter_output>Collecting tensorflow-gpu
[?25l  Downloading https://files.pythonhosted.org/packages/1d/a2/5ccf0a418eb22e0a2ae9edc1e7f5456d0a4b8b49524572897564b4030a9b/tensorflow_gpu-2.5.0-cp37-cp37m-manylinux2010_x86_64.whl (454.3MB)
[K     |████████████████████████████████| 454.3MB 41kB/s 
[?25hRequirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu) (3.12.4)
Requirement already satisfied: six~=1.15.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu) (1.15.0)
Requirement already satisfied: wrapt~=1.12.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu) (1.12.1)
Requirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu) (0.36.2)
Requirement already satisfied: opt-einsum~=3.3.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu) (3.3.0)
Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu) (1.1.0)
Requ[...]<jupyter_text>Data instead of just inception-v3, implement with cgg-16 as well<jupyter_code>## required libraries
import tensorflow as tf
from tensorflow.keras.layers import Input, Lambda, Dense, Flatten
from tensorflow.keras.models import Model
from tensorflow.keras.applications.vgg16 import VGG16 ## check it's arch, a 42 layer layer net
from tensorflow.keras.applications.vgg16 import preprocess_input
from tensorflow.keras.preprocessing import image
from tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img
from glob import glob
import matplotlib.pyplot as plt

import numpy as np
import sys

import pandas as pd
print(tf.__version__)

from google.colab import drive
drive.mount('/content/drive/')
dataDir = 'drive/MyDrive/git/Datasets/tomato/'
## see if we are at right loc
!ls drive/MyDrive/git/Datasets/tomato/<jupyter_output>train  valid<jupyter_text>work<jupyter_code>## will need to resize all images (if ot so) 
IMAGE_SIZE = [224,224]
train_path = dataDir+'train'
valid_path = dataDir+'valid'<jupyter_output><empty_output><jupyter_text>_importing the inception-v3 lib. & adding preprocessing layer to the front of inception v3_  _here it's done using imagenet weights_<jupyter_code>inception = VGG16(input_shape=IMAGE_SIZE + [3], weights='imagenet', include_top=False)
# don't traing existing weights
for layer in inception.layers:
  layer.trainable = False

## for getting number of unique output class
folders = glob(train_path + '/*')
len(folders), folders<jupyter_output><empty_output><jupyter_text>_so, we have 10 classes_<jupyter_code># adding layers
x = Flatten()(inception.output)
prediction = Dense(len(folders), activation='softmax')(x)

## creating a model object
model = Model(inputs=inception.input, outputs=prediction)
# view the structure of the model
model.summary()
model.compile(
    loss='categorical_crossentropy',
    optimizer='adam',
    metrics=['accuracy']
)
## using the imgages data generator 
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range=0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)
## training  ---  providing same target size as that for image size
training_set = train_datagen.flow_from_directory(train_path,
                                                 target_size = (224,224),
                                                 batch_size = 32,
                                                 class_mode = 'categorical')
test_set = test_datagen.flow_from_directory(valid_path,
                                                 target_size = (224,224),
                                                 batch_size = 32,
                                                 class_mode = 'categorical')
## fit the model - a time consuming process
model = tf.keras.models.load_model('/content/drive/MyDrive/my_model.h5')#0 done
for i in range(10):
  r = model.fit(
      training_set,
      validation_data=test_set,
      steps_per_epoch=100, #0,
      epochs=1, #10,
      validation_steps=100
      #steps_per_epoch=len(training_set),
      #validation_steps=len(test_set)
      )
  model.save('/content/drive/MyDrive/my_model.h5')
  model = tf.keras.models.load_model('/content/drive/MyDrive/my_model.h5')<jupyter_output>100/100 [==============================] - 4021s 40s/step - loss: 0.4534 - accuracy: 0.8491 - val_loss: 0.4971 - val_accuracy: 0.8291
100/100 [==============================] - 3285s 33s/step - loss: 0.3907 - accuracy: 0.8572 - val_loss: 0.5006 - val_accuracy: 0.8331
100/100 [==============================] - 3204s 32s/step - loss: 0.3567 - accuracy: 0.8781 - val_loss: 0.4892 - val_accuracy: 0.8438<jupyter_text>**Here model can be seen running only 3 times because, it was ran 7 more times prior to this final execution.**  Performance<jupyter_code>## model accuracy
print("Accuracy: ",r.history['accuracy'])<jupyter_output>Accuracy:  [0.878125011920929]
{'fraction_non_alphanumeric': 0.1298972993533663, 'fraction_numerical': 0.058577405857740586, 'mean_word_length': 3.5611448395490024, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '456082', 'license': 'MIT', 'n_tokens_mistral': 1909, 'n_tokens_neox': 1711, 'n_words': 385, 'path': 'Labs/deepNN_lab5.ipynb', 'repo_name': 'ravi-prakash1907/Machine-Learning-for-Cyber-Security', 'size': '5258'}