<jupyter_start><jupyter_text>Copyright 2018 The TensorFlow Authors.<jupyter_code>#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#@title MIT License
#
# Copyright (c) 2017 <NAME>
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.<jupyter_output><empty_output><jupyter_text>Basic classification: Classify images of clothing       View on TensorFlow.org        Run in Google Colab        View source on GitHub        Download notebook   This guide trains a neural network model to classify images of clothing, like sneakers and shirts. It's okay if you don't understand all the details; this is a fast-paced overview of a complete TensorFlow program with the details explained as you go.This guide uses [tf.keras](https://www.tensorflow.org/guide/keras), a high-level API to build and train models in TensorFlow. This block of code created to check if Google colab has the new version of Tensor Flow<jupyter_code>try:
  %tensorflow_version 2.x
except Exception:
  pass

# This line of code used to write a Python 2/3 compatible codebase
from __future__ import absolute_import, division, print_function, unicode_literals

# Import TensorFlow library
import tensorflow as tf
# Import keras from TensorFlow library
from tensorflow import keras

# Import numpy and matplotlib.pyplot as helper libraries
import numpy as np
import matplotlib.pyplot as plt

# check the imported version of tensorflow
print(tf.__version__)<jupyter_output>2.0.0<jupyter_text>Import the Fashion MNIST dataset For our program we will use [Fashion MNIST](https://github.com/zalandoresearch/fashion-mnist) dataset which contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels), as seen here:60000 images from this dataset will be used to train our network and 10000 to check how our algorithm works       <img src="https://tensorflow.org/images/fashion-mnist-sprite.png"         alt="Fashion MNIST sprite"  width="600">        Figure 1. Fashion-MNIST samples (by Zalando, MIT License).&nbsp;<jupyter_code># import fashion MNIST datset directly from tensorflow
fashion_mnist = keras.datasets.fashion_mnist

# load data from fashion_mnist dataset and split it on 4 arrays
#train_image and train_labels will be used as trainig set, test_images and test_labels as test set.
# they all arrays
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()<jupyter_output><empty_output><jupyter_text>The images arrays are 28x28 NumPy arrays each element of it represent as pixl values from 0 to 255.The labels arrays = arrays of integeres from 0 to 9 which represent the clothing class      Label    Class        0    T-shirt/top        1    Trouser          2    Pullover          3    Dress          4    Coat          5    Sandal          6    Shirt          7    Sneaker          8    Bag          9    Ankle boot<jupyter_code># just wanted to check how the images array looks like
train_images<jupyter_output><empty_output><jupyter_text>Each image is mapped to a single label. Class nams are not included with the dataset this why we are storing them in the class_names array to use later<jupyter_code>class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']<jupyter_output><empty_output><jupyter_text>Explore the dataWe are need to check the format of the each set array that we will use to train and test our model. To do this we're using .shape property to get information about each of the array. The following shows there are 60,000 images in the training set, with each image represented with array of 28 x 28 pixels:<jupyter_code>train_images.shape<jupyter_output><empty_output><jupyter_text>There are also 60000 labels in training set:<jupyter_code>len(train_labels)<jupyter_output><empty_output><jupyter_text>Each label is an integer between 0 and 9:<jupyter_code>train_labels<jupyter_output><empty_output><jupyter_text>We are doing the same of the test set. There are 10 000 images in test set that represent as array of 28 x 28 pixels<jupyter_code>test_images.shape<jupyter_output><empty_output><jupyter_text>And the test set contains 10,000 images labels:<jupyter_code>len(test_labels)<jupyter_output><empty_output><jupyter_text>Preprocess the dataThe data must be preprocessed before training the network. If you inspect the first image in the training set, you will see that the pixel values fall in the range of 0 to 255:<jupyter_code># check the first image in train_images array
train_images[0]
# The purpose of using plt.figure() is to create a figure object using matplotlib library
plt.figure()
# show image from the pixel values of the first element in train_images array
plt.imshow(train_images[0])
# display colorbar from the right of the figure, because 
plt.colorbar()
# don't show grid lines
plt.grid(False)
#I am not sure why do we need plt.show() here. It seems like everything works fien without it
plt.show()<jupyter_output><empty_output><jupyter_text>We need to change our pixel data to be in range of 0 to 1. Todo this we need to  divide all the values by 255.<jupyter_code>train_images = train_images / 255.0

# We do the same for the training set and test set so thay can be processed the same way.
test_images = test_images / 255.0<jupyter_output><empty_output><jupyter_text>To see that all data changed correctly we display first 25 images with labels under them<jupyter_code># create a figure that has size 10 x 10 inches
plt.figure(figsize=(10,10))

#create a for loop that wiil be repeated 25 times one time for each image that we want to draw
for i in range(25):
#pyplot.subplots creates a figure and a grid of subplots with a single call. Number of rows/columns of the subplot grid is 5. 
# The third parameter is index of the current subplot figure?

    plt.subplot(5,5,i+1)
    
#     Disable xticks (numbers on x-axis):
    plt.xticks([])
#    Disable yticks(numbers on y-axis):
    plt.yticks([])
#     don't display grid
    plt.grid(False)
    
#     Display image form the train_images array in the greyscale colors
    plt.imshow(train_images[i], cmap=plt.cm.binary)
#     display label of the image from the class names labels on the x-axis
    plt.xlabel(class_names[train_labels[i]])
plt.show()<jupyter_output><empty_output><jupyter_text>Build the modelBuilding the neural network requires configuring the layers of the model, then compiling the model.  Set up the layersThe basic building block of a neural network is the *layer*. Layers extract representations from the data fed into them. Hopefully, these representations are meaningful for the problem at hand.Most of deep learning consists of chaining together simple layers. Most layers, such as `tf.keras.layers.Dense`, have parameters that are learned during training.<jupyter_code># Build the Sequential model object(wich is a linear stack of layers) with the given layers

model = keras.Sequential([
#     we fltten the data. ieach emge is consists of an array of 28x28pixel consist after flattening it will become
#     a one-dimensional array with 784 pixels.
    keras.layers.Flatten(input_shape=(28, 28)),
#     define input and a hidden layer with 128 neurons and an activation relu function. 128 is dimensionality of the output space. Dense is 2D layer.
#     relu stands for Rectified Linear Unit. it returns tensor. relu fi=unction looks like this f(x) = max(0,x)
#     using different activations function doesn’t affect what our network can learn, only how fast (how many data/epochs it needs).
#     But the issue is that all the negative values become zero immediately which decreases the ability of the model to fit or train from the data properly.
#     That means any negative input given to the ReLU activation function turns the value into zero immediately in the graph
#     but in our dataset we have only positive numbers so we can use relu
    keras.layers.Dense(128, activation='relu'),
#     build the output layer with softmax function  It normalizes our output in a non-linear fashion so that the sum of output for all classes is equals to 1
#     it consists of 10-neurons.Each node contains a score that indicates the probability that the current image belongs to one of the 10 classes.
    keras.layers.Dense(10, activation='softmax')
])<jupyter_output><empty_output><jupyter_text>Compile the model<jupyter_code># Before start trainig our model we need to compile it first, by adding some new parameters we tell our neural network 
# how to start the training process.
# adam stands for Adam: Adaptive moment estimation. the adam has relatively low memory requirements 
model.compile(optimizer='adam',
#   we are targeting are integers, this is why use of sparse_categorical_crossentropy
#   this is the only thing that I found about this parameter that I can understand for now 
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code># we feed our trainig data(train_images and train_labels) to our model with "fit" method
# the model learns how to associate images with labels
# in each epochs(round) model accuracy improves. total we have 10 epochs
model.fit(train_images, train_labels, epochs=10)<jupyter_output>Train on 60000 samples
Epoch 1/10
60000/60000 [==============================] - 5s 82us/sample - loss: 3.0746 - accuracy: 0.6955
Epoch 2/10
60000/60000 [==============================] - 4s 74us/sample - loss: 0.6938 - accuracy: 0.7556
Epoch 3/10
60000/60000 [==============================] - 4s 66us/sample - loss: 0.5788 - accuracy: 0.8042
Epoch 4/10
60000/60000 [==============================] - 4s 68us/sample - loss: 0.5152 - accuracy: 0.8231
Epoch 5/10
60000/60000 [==============================] - 4s 66us/sample - loss: 0.4961 - accuracy: 0.8305
Epoch 6/10
60000/60000 [==============================] - 4s 67us/sample - loss: 0.4848 - accuracy: 0.8354
Epoch 7/10
60000/60000 [==============================] - 4s 71us/sample - loss: 0.4726 - accuracy: 0.8393
Epoch 8/10
60000/60000 [==============================] - 4s 68us/sample - loss: 0.4678 - accuracy: 0.8411s - loss: 0.4664 - accuracy: 
Epoch 9/10
60000/60000 [==============================] - 5s 83us/sample - loss: 0.4582 - ac[...]<jupyter_text>On the trainig data our model reaches accuracy about 88%  Evaluate accuracyAfter training process finished, we need to check how our model performs on the test dataset:<jupyter_code># Verbose is verbossity mode and I don't know what it means and why it is 2. in keras documentation it says it can be only 0 or 1 https://keras.io/models/model/
test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)
# print our test accuracy
print('\nTest accuracy:', test_acc)<jupyter_output>10000/1 - 0s - loss: 0.3903 - accuracy: 0.8227

Test accuracy: 0.8227<jupyter_text>Our model accuracy on test dataset is leess than on out trainig set.When machine learning moel performs worse on the new dataset than on training data It calls overfitting  Make predictionsNow when our model trained we can use it to make predictions on some images<jupyter_code>predictions = model.predict(test_images)<jupyter_output><empty_output><jupyter_text>predictions is array of array which holds numbers that represent confidence level of the model about the label name for the current test element.<jupyter_code>predictions[0]<jupyter_output><empty_output><jupyter_text>We checked the first element in predictions array.This is is an array of 10 numbers. The last label which represents ankle boot has the highest confidence level 9.7308683e-01<jupyter_code># return the indices of the maximum values along an axis
np.argmax(predictions[0])<jupyter_output><empty_output><jupyter_text>So, the model is most confident that this image is an ankle boot, or `class_names[9]`. Examining the test label shows that this classification is correct:<jupyter_code>test_labels[0]<jupyter_output><empty_output><jupyter_text>Graph this to look at the full set of 10 class predictions.<jupyter_code># create function to create the image from the given array
def plot_image(i, predictions_array, true_label, img):
  predictions_array, true_label, img = predictions_array, true_label[i], img[i]

# get rid of tick and grid
  plt.grid(False)
  plt.xticks([])
  plt.yticks([])

# show image in grayscale
  plt.imshow(img, cmap=plt.cm.binary)

# if label was guessed right use blue color otherwise red
  predicted_label = np.argmax(predictions_array)
  if predicted_label == true_label:
    color = 'blue'
  else:
    color = 'red'

# function to create labe for the x-axis following the oattern
  plt.xlabel("{} {:2.0f}% ({})".format(class_names[predicted_label],
                                100*np.max(predictions_array),
                                class_names[true_label]),
                                color=color)

# function to create value graph according to the data  
def plot_value_array(i, predictions_array, true_label):
    
  predictions_array, true_label = predictions_array, true_label[i]
# delete grid
  plt.grid(False)
# numbers on x-axis from 0 to 9
  plt.xticks(range(10))
# get rif of numbers on y-axis
  plt.yticks([])
# make the plotbar graph 
  thisplot = plt.bar(range(10), predictions_array, color="#777777")
  plt.ylim([0, 1])
  predicted_label = np.argmax(predictions_array)
#     if prediced label is different than true label set color to red
  thisplot[predicted_label].set_color('red')
  thisplot[true_label].set_color('blue')<jupyter_output><empty_output><jupyter_text>Let's look at the 0th image, predictions, and prediction array. Correct prediction labels are blue and incorrect prediction labels are red. The number gives the percentage (out of 100) for the predicted label.<jupyter_code># first image from rredictions array
i = 0
# build the figure 6x3 inch size
plt.figure(figsize=(6,3))
# create subplot with image using plot image function
plt.subplot(1,2,1)
plot_image(i, predictions[i], test_labels, test_images)
# create subplot with plotbar using plot_value_array function
plt.subplot(1,2,2)
plot_value_array(i, predictions[i],  test_labels)
plt.show()
i = 12
plt.figure(figsize=(6,3))
plt.subplot(1,2,1)
plot_image(i, predictions[i], test_labels, test_images)
plt.subplot(1,2,2)
plot_value_array(i, predictions[i],  test_labels)
plt.show()<jupyter_output><empty_output><jupyter_text>Let's plot several images with their predictions. Note that the model can be wrong even when very confident.<jupyter_code># Plot more images
num_rows = 5
num_cols = 3
num_images = num_rows*num_cols
plt.figure(figsize=(2*2*num_cols, 2*num_rows))
for i in range(num_images):
  plt.subplot(num_rows, 2*num_cols, 2*i+1)
  plot_image(i, predictions[i], test_labels, test_images)
  plt.subplot(num_rows, 2*num_cols, 2*i+2)
  plot_value_array(i, predictions[i], test_labels)
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>Finally, use the trained model to make a prediction about a single image.<jupyter_code># Grab an image from the test dataset.
img = test_images[1]

print(img.shape)<jupyter_output>(28, 28)<jupyter_text>`tf.keras` models are optimized to make predictions on a *batch*, or collection, of examples at once. So w need to make our image to a list so we can check our model make predictions on it<jupyter_code># Add the image to a batch where it's the only member.
img = (np.expand_dims(img,0))

print(img.shape)<jupyter_output>(1, 28, 28)<jupyter_text>Now predict the correct label for this image using model.predict:<jupyter_code>predictions_single = model.predict(img)

print(predictions_single)
# and build a bar plot to display our data. the label tha moel was most confident about was pulover which is correct
plot_value_array(1, predictions_single[0], test_labels)
_ = plt.xticks(range(10), class_names, rotation=45)<jupyter_output><empty_output><jupyter_text>`model.predict` returns a list of lists—one list for each image in the batch of data. Grab the predictions for our (only) image in the batch:<jupyter_code>np.argmax(predictions_single[0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08759413396749902, 'fraction_numerical': 0.029556650246305417, 'mean_word_length': 4.721412374473599, 'pattern_counts': {'":': 0, '<': 110, '<?xml version=': 0, '>': 110, 'https://': 5, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '442667', 'license': 'MIT', 'n_tokens_mistral': 5560, 'n_tokens_neox': 5087, 'n_words': 2116, 'path': 'classification.ipynb', 'repo_name': 'nastinsk/clothing-classifier', 'size': '17671'}