<jupyter_start><jupyter_text>Categorical Classification With TensorflowYou want to classify your data into different categories.A good exmaple of this would be recognising different animals in an image - e.g. Cat, Dog, Horse, Cow, Sheep.In this notebook we'll be extending the work from Binary Classification notebook to recognise more than two shapes.<jupyter_code># Standard set of imports
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D
from tensorflow.data import Dataset
import numpy as np
import matplotlib.pyplot as plt
# some constants for use later
image_width = 100
image_height = 100
batch_size = 30<jupyter_output><empty_output><jupyter_text>Training DataWe're going to generate some dummy training data. We'll generate some simple images that are blank, contain a square or a circle or a triangle. Our model will try and predict what class the image falls into.<jupyter_code>labels=["blank","square","circle","triangle"]<jupyter_output><empty_output><jupyter_text>Data GeneratorThis is an infinite generator that will generate a pair of input and outpus values.There's an important difference from our previous generator - we are now returing a one hot encoded value for our label using `tf.one_hot`.For our different labels this will generate the following labels:```blank =    [1, 0, 0, 0]square =   [0, 1, 0, 0]circle =   [0, 0, 1, 0]triangle = [0, 0, 0, 1]```<jupyter_code>def data_generator():
    i = 0
    while(True):
        if i >= 1000:
            i = 0
        # our output value will be the one hot encoded version of: 0,1,2,3 - corresponding to our labels - "blank", "square", "circle", "triangle"
        Y = i % 4
        X = np.zeros((image_width, image_height, 1))
        
        # size of our shape
        radius = int(np.random.uniform(10,20))
        # position of our shape
        center_x = int(np.random.uniform(radius, image_width - radius))
        center_y = int(np.random.uniform(radius, image_height - radius))
        
        if Y == 1: # generate a square
            X[center_y - radius:center_y + radius, center_x - radius:center_x + radius] = 1        
        elif Y == 2: # generate a circle
            for y in range(-radius, radius):
                for x in range(-radius, radius):
                    if x*x + y*y <= radius*radius:
                        X[y+center_y, x+center_x] = 1
        elif Y==3:
            for y in range(-radius, radius):
                for x in range(-radius, radius):
                    if abs(x) < (y+radius)/2:
                        X[y+center_y, x+center_x] = 1            
        else: # blank image
            pass
        yield X, tf.one_hot(Y, 4)
        i = i + 1<jupyter_output><empty_output><jupyter_text>DatasetThis creates a dataset that we can feed into the tensorflow fit function for training<jupyter_code>train_dataset = tf.data.Dataset.from_generator(
    data_generator, 
    output_types = (tf.float32, tf.int32),
    output_shapes=((image_width, image_height, 1), (4))
)
train_dataset = train_dataset.batch(batch_size=batch_size)<jupyter_output><empty_output><jupyter_text>Visualising our training dataWe can display a selection of images from our training data along with their labels.<jupyter_code>import itertools

def plot_images(X, Y):
    fig, axes = plt.subplots(2, 5, figsize=(10, 5))
    axes = axes.flatten()
    for x, y, ax in zip(X, tf.cast(Y, tf.float32), axes):
        # show the image
        ax.imshow(x)
        # work out the label
        label = labels[int(y)]
        # show the label beneath the plot
        ax.text(10, image_height+10, f"{label}", bbox={'facecolor': 'white', 'pad': 5})
        # turn off the axis to make the display a bit cleaner
        ax.axis("off")
    plt.tight_layout()
    plt.show()

# get a batch of images and labels from our dataset
X, Y = next(iter(train_dataset))
# and show the images along with the label - not the use of nump() to get the values from the tensor
plot_images(X, tf.argmax(Y, axis=1).numpy())<jupyter_output><empty_output><jupyter_text>Our very simple modeWe don't need a very complicated model for our problem, so we'll just define a small convolutional neural network with a hidden layer and an output layer. Our output layer now has 4 neurons as we have 4 different classes to predict.It's important that the activation function should be `softmax`. This activation function will make sure that the output values<jupyter_code>model = Sequential([
    Conv2D(8, 3, 
           padding='same',
           activation='relu',
           input_shape=(image_width, image_height, 1),
           name='conv_layer'),
    MaxPooling2D(name='max_pooling'),
    Flatten(),
    Dense(
        10,
        activation='relu',
        name='hidden_layer'
    ),
    Dense(4, activation='softmax', name='output')
])<jupyter_output><empty_output><jupyter_text>Compile our modelFor our loss function we need to use `CategoricalCrossentropy`.Crossentropy quantifies the difference between two probability distribution.We have more than two classes so need to use CategoricalCrossentropy for our loss function.<jupyter_code>model.compile(optimizer='adam',
              loss=tf.keras.losses.CategoricalCrossentropy(),
              metrics=['accuracy'])

model.summary()<jupyter_output><empty_output><jupyter_text>Fitting our modelWe can now fit our model to the data. We have a very simple model to solve so we don't need to run for very many epochs to get a good fit.<jupyter_code>history = model.fit(
    train_dataset,
    steps_per_epoch=200,
    epochs=10
)<jupyter_output><empty_output><jupyter_text>Testing our model with some examplesYou should end up with an accuracy approaching > 90%. Longer training should give you something approaching 100%.Let's generate some more examples and see what the output of the model is.<jupyter_code># get a batch of samples from the dataset
X, _ = next(iter(train_dataset))
# ask the model to predict the output for our samples
predicted_Y = model.predict(X.numpy())
# work out the max indices
max_indices = tf.argmax(predicted_Y, axis = 1)
# show the images along with the predicted value
plot_images(X, max_indices)
# set the format to 2 decimal places
np.set_printoptions(formatter={'float': lambda x: "{0:0.2f}".format(x)})
predicted_Y[:10]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09238705643270283, 'fraction_numerical': 0.01516335782397999, 'mean_word_length': 3.5056338028169014, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '573816', 'license': 'CC0-1.0', 'n_tokens_mistral': 1973, 'n_tokens_neox': 1872, 'n_words': 683, 'path': 'Categorical Classification With TensorFlow.ipynb', 'repo_name': 'atomic14/tensorflow-tutorial', 'size': '6397'}