<jupyter_start><jupyter_text>Copyright 2019 The TensorFlow Authors. 2020 <NAME><jupyter_code>#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.<jupyter_output><empty_output><jupyter_text>NoticeRemember to enable GPU to make everything run faster (Runtime -> Change runtime type -> Hardware accelerator -> GPU).Also, if you run into trouble, simply reset the entire environment and start from the beginning:*   Edit -> Clear all outputs*   Runtime -> Reset all runtimes  Lab 04a: Dogs vs Cats Image Classification Without Image Augmentation       Run in Google Colab        View source on GitHub   In this tutorial, we will discuss how to classify images into pictures of cats or pictures of dogs. We'll build an image classifier using a `keras_model_sequential()` model and load data using `keras` function `image_data_generator()`. Specific concepts that will be covered:In the process, we will build practical experience and develop intuition around the following concepts* Building _data input pipelines_ using the `image_data_generator()` class — How can we efficiently work with data on disk to interface with our model?* _Overfitting_ - what is it, how to identify it?**Before you begin**Before running the code in this notebook, reset the runtime by going to **Runtime -> Reset all runtimes** in the menu above. If you have been working through several notebooks, this will help you avoid reaching Colab's memory limits.  Install and load dependenciesFirst, you'll need to install and load R package Keras which will also install TensorFlow. We'll also install package `fs` which has useful functionality for working with our filesystem.<jupyter_code>install.packages(c("keras", "fs"))
library(keras)<jupyter_output><empty_output><jupyter_text>Data Loading To build our image classifier, we begin by downloading the dataset. The dataset we are using is a filtered version of Dogs vs. Cats dataset from Kaggle (ultimately, this dataset is provided by Microsoft Research).In previous Colabs, we've used TensorFlow Datasets, which is a very easy and convenient way to use datasets. In this Colab however, we will make use of the class `image_data_generator()` which will read data from disk. We therefore need to directly download *Dogs vs. Cats* from a URL and unzip it to the Colab filesystem.<jupyter_code>URL <- "https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip"
zip_dir <- get_file('cats_and_dogs_filtered.zip', origin = URL, extract = TRUE)<jupyter_output><empty_output><jupyter_text>The dataset we have downloaded has the following directory structure.cats_and_dogs_filtered|__ train    |______ cats: [cat.0.jpg, cat.1.jpg, cat.2.jpg ...]    |______ dogs: [dog.0.jpg, dog.1.jpg, dog.2.jpg ...]|__ validation    |______ cats: [cat.2000.jpg, cat.2001.jpg, cat.2002.jpg ...]    |______ dogs: [dog.2000.jpg, dog.2001.jpg, dog.2002.jpg ...]We can list the directories with the following terminal command:<jupyter_code>zip_dir_base <- dirname(zip_dir)
fs::dir_tree(zip_dir_base, recurse = 2)<jupyter_output><empty_output><jupyter_text>We'll now assign variables with the proper file path for the training and validation sets.<jupyter_code>base_dir <- fs::path(zip_dir_base, "cats_and_dogs_filtered")
train_dir <- fs::path(base_dir, "train")
validation_dir <- fs::path(base_dir, "validation")

train_cats_dir <- fs::path(train_dir, "cats")
train_dogs_dir <- fs::path(train_dir, "dogs")
validation_cats_dir <- fs::path(validation_dir, "cats")
validation_dogs_dir <- fs::path(validation_dir, "dogs")<jupyter_output><empty_output><jupyter_text>Understanding our data Let's look at how many cats and dogs images we have in our training and validation directory<jupyter_code>num_cats_tr <- length(fs::dir_ls(train_cats_dir))
num_dogs_tr <- length(fs::dir_ls(train_dogs_dir))

num_cats_val <- length(fs::dir_ls(validation_cats_dir))
num_dogs_val <- length(fs::dir_ls(validation_dogs_dir))

total_train <- num_cats_tr + num_dogs_tr
total_val <- num_cats_val + num_dogs_val
cat('total training cat images:', num_cats_tr, "\n")
cat('total training dog images:', num_dogs_tr, "\n")

cat('total validation cat images:', num_cats_val, "\n")
cat('total validation dog images:', num_dogs_val, "\n")
cat("--", "\n")
cat("Total training images:", total_train, "\n")
cat("Total validation images:", total_val, "\n")<jupyter_output><empty_output><jupyter_text>Setting Model Parameters For convenience, we'll set up variables that will be used later while pre-processing our dataset and training our network.<jupyter_code>batch_size = 100  # Number of training examples to process before updating our models variables
img_shape  = 150  # Our training data consists of images with width of 150 pixels and height of 150 pixels<jupyter_output><empty_output><jupyter_text>Data Preparation  Images must be formatted into appropriately pre-processed floating point tensors before being fed into the network. The steps involved in preparing these images are:1. Read images from the disk2. Decode contents of these images and convert it into proper grid format as per their RGB content3. Convert them into floating point tensors4. Rescale the tensors from values between 0 and 255 to values between 0 and 1, as neural networks prefer to deal with small input values.Fortunately, all these tasks can be done using the class [**`image_data_generator()`**](https://keras.rstudio.com/reference/image_data_generator.html).We can set this up in a couple of lines of code.<jupyter_code>train_image_generator <- image_data_generator(rescale = 1/255)
validation_image_generator <- image_data_generator(rescale = 1/255)<jupyter_output><empty_output><jupyter_text>After defining our image generators, which dictate the processing of our training and validation images, [**flow_from_directory**](https://keras.rstudio.com/reference/flow_images_from_directory.html) method creates a **data generator** which will load images from the disk, apply rescaling, and resize them using single line of code.At this stage we define how we want our images to flow from the directory (e.g. through bacth size), image properties and the class mode of our data (which is importtant information that helps the data generator create appropriate labels for our images according to our folder's file structure).As we only have two classes of images, we need to set `class_mode` to `"binary"`.<jupyter_code>train_data_gen <- flow_images_from_directory(directory = train_dir,
                                             generator = train_image_generator,
                                             target_size = c(img_shape, img_shape),
                                             class_mode = "binary",
                                             batch_size = batch_size)
val_data_gen <- flow_images_from_directory(directory = validation_dir,
                                             class_mode = "binary",
                                             generator = validation_image_generator,
                                             target_size = c(img_shape, img_shape),
                                             batch_size = batch_size)<jupyter_output><empty_output><jupyter_text>Exploring the data through the Image generator We can get information about our data generators by calling properties associated with them, such as `class_mode`, `labels` or `class_indices`<jupyter_code>val_data_gen$class_mode
val_data_gen$labels
train_data_gen$class_indices
val_data_gen$class_indices<jupyter_output><empty_output><jupyter_text>We can get the next batch of data from a data generator using `generator_next()`, in this case the first bacth will be returned.<jupyter_code>sample_training_images <- generator_next(train_data_gen)
str(sample_training_images)<jupyter_output><empty_output><jupyter_text>We can also get data by indexing a data generator (note it is zero indexed, so to get the first batch you index with 0, because the underlying code it is running is python).<jupyter_code>sample_training_images <- train_data_gen[0]
str(sample_training_images)<jupyter_output><empty_output><jupyter_text>We can see the structure of our batched, generated data is a list with two elements. - The first element contains our **features (images)** in a 4 dimensional array (batch number, height, width, channels). As we are dealing with RGB images, this time each image has 3 channels  - The second element contains our **targets (labels)** and is a 1d vector. This is because we set the `class_mode` to `"binary"`. If we'd left it as the dafault (`"categorical"`), it would have generated [**one-hot encoded**](https://www.youtube.com/watch?v=BecEHOVmx9o) targets (a matrix of dimensions *batch number* x *classes*) with the correct target class encoded with 1 and all other classes with 0. It's important to check for the shape of the targets as it affects the choice and effectiveness of the loss function when we compile the model.<jupyter_code>plot_rgb_image <- function(image_array){
  image_array %>%
  array_reshape(dim = c(dim(.)[1:3])) %>%
  as.raster(max = 1) %>%
  plot()
}

sample_training_images[[1]][1,,,] %>%
  plot_rgb_image()
# set plotting options
options(repr.plot.width = 16, repr.plot.height = 4)

# set number of images to plot
n <- 4

# Loop plotting over n images
layout(matrix(1:n, ncol = 4), respect = FALSE)
for(i in 1:n){
  sample_training_images[[1]][i,,,] %>%
  plot_rgb_image()
}<jupyter_output><empty_output><jupyter_text>Model Creation  Exercise 4.1  Define the model (20 mins for 4.1, 4.2 & 4.3)The model consists of four convolution blocks with a max pool layer in each of them. Then we have a fully connected layer with 512 units, with a `relu` activation function. The model will output class probabilities for two classes — dogs and cats — using `softmax`. The list of model layers:* 2D Convolution - `img_shape` x `img_shape` x 3 input shape, 32 filters, 3x3 kernel, ReLU activation* 2D Max pooling - 2x2 kernel* 2D Convolution - 64 filters, 3x3 kernel, ReLU activation* 2D Max pooling - 2x2 kernel* 2D Convolution - 128 filters, 3x3 kernel, ReLU activation* 2D Max pooling - 2x2 kernel* 2D Convolution - 128 filters, 3x3 kernel, ReLU activation* 2D Max pooling - 2x2 kernel* Flatten* Dense - 512 nodes, ReLU activation* Dense - 2 nodes, softmax activationCheck the documentation on [Guide to Keras Basics](https://keras.rstudio.com/articles/guide_keras.html) and [Keras functions](https://keras.rstudio.com/reference/index.html) for information on how to specify the layers.<jupyter_code># TODO - Create the CNN model as specified above<jupyter_output><empty_output><jupyter_text>Exercise 4.1 SolutionThe solution for the exercise can be found [here](https://colab.research.google.com/github/rses-dl-course/rses-dl-course.github.io/blob/master/notebooks/R/solutions/E.4.1.ipynb)  Model SummaryLet's look at all the layers of our network. We can view a summary of our network by simply printing the model to the console.<jupyter_code>model<jupyter_output><empty_output><jupyter_text>Exercise 4.2 Compile the modelAs usual, we will use the `adam` optimizer. Since we output a softmax categorization, we'll use `sparse_categorical_crossentropy` as the loss function. We would also like to look at training and validation accuracy on each epoch as we train our network, so we are passing in the metrics argument.<jupyter_code># TODO - Compile the model<jupyter_output><empty_output><jupyter_text>Exercise 4.2 SolutionThe solution for the exercise can be found [here](https://colab.research.google.com/github/rses-dl-course/rses-dl-course.github.io/blob/master/notebooks/R/solutions/E.4.2.ipynb)  Exercise 4.3 Train the model It's time we train our network. Have a look at the [documentation](https://keras.rstudio.com/reference/fit.html) to refresh how to use the `fit` function.* Since our data are coming from a training data generator, we provide that as our `x` argument.* Since we have a validation dataset, we can use this to evaluate our model as it trains by supplying the validation data generator  to argument `validation_data`. Fit your model for 40 epochs. Remember to assign it to a `history` object.<jupyter_code># TODO - Fit the model<jupyter_output><empty_output><jupyter_text>Exercise 4.3 SolutionThe solution for the exercise can be found [here](https://colab.research.google.com/github/rses-dl-course/rses-dl-course.github.io/blob/master/notebooks/R/solutions/E.4.3.ipynb) Let's have a look at our history<jupyter_code>history<jupyter_output><empty_output><jupyter_text>Visualizing results of the training We'll now visualize the results we get after training our network.<jupyter_code>plot(history)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08427692073865796, 'fraction_numerical': 0.01261494034501102, 'mean_word_length': 4.7897052353717555, 'pattern_counts': {'":': 0, '<': 102, '<?xml version=': 0, '>': 89, 'https://': 11, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 19, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '565625', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 3913, 'n_tokens_neox': 3643, 'n_words': 1452, 'path': 'notebooks/R/R04_C01_dogs_vs_cats_without_augmentation.ipynb', 'repo_name': 'rses-dl-course-durham/rses-dl-course-durham.github.io', 'size': '13166'}