<jupyter_start><jupyter_text>How To mesure beauty 1. Nombre d'or>  - La largeur de la bouche est 1.618 fois la largeur du nez>  - La largeur de la bouche est 1.618 fois l'intervalle qui la separe de l'extremité du visage>  - Le nez etant un triangle, les cotés font 1.618 fois la base 2. Facial Keypoint Detection (Dataset)>  - Le dataset contient 15 FKPs (paper)>  - Le dataset contient 68 FKPs (an other dataset)>  - Chaque FKP a deux coordonnées spatiales (x, y)  Targered FKPs MOUTH    : 49 - 55 NOSE>  PEEK   : 28>  BASE   : 32 - 36 EDGES>  LEFT   : 4>  RIGHT  : 14  WORKFLOW 1. Data Preprocessing>  - Download FKPs dataset>  - Load and visualize data>  - Data Transformation (Normalization, Rotation, Rescal, RandomCrop)>  - Data Iteration and Batching (training & testing)>  ```python [[IMAGE ARRAY], [X, Y]]```>  2 Possibilites> 1.  Entrainer 68 CNN qui predisent les coordonnees (x, y) d'un FKP - ```python [[IMAGE ARRAY], [X, Y]]```> 2.  Entrainer 1 CNN qui predit les 136 coordonnees- ```python [[IMAGE ARRAY], [X..., Y...]]``` 2. Defining and Training a Naimish CNN to predict FKPs>  - Define model with Keras>  - Define Training Parameters (Epoch, Loss Function, Optimizer)>  - Train model>  - Test model 3. Defining a Face Beauty classifier>  1 Mean Square-  Calcul de la moyenne quadratique>  3.1 La largeur de la bouche est 1.618 fois la largeur du nez-  mouthWidth = distance(FKP49, FKP55)-  noseWIdth = distance(FKP32, FKP36)-  ratioMouthNose = mouthWidth / noseWIdth-  accuracy (taux d'exactitude) = ratioMouthNose / 1.618 = 1.>  3.2 La largeur de la bouche est 1.618 fois l'intervalle qui la separe de l'extremité du visage-  mouthWidth = distance(FKP49, FKP55),-  leftMouthEdgeInterval = distance(FKP4, FKP49)-  rightMouthEdgeInterval = distance(FKP55, FKP14)-  Left Mouth-  ratioLeftMouthEdgeInterval = mouthWidth / leftMouthEdgeInterval-  leftAccuracy (taux d'exactitude) = ratioLeftMouthEdgeInterval / 1.618 = 1.-  Right Mouth-  ratioRightMouthEdgeInterval = mouthWidth / rightMouthEdgeInterval-  rightAccuracy (taux d'exactitude) = ratioRightMouthEdgeInterval / 1.618 = 1.>  3.3 Le nez etant un triangle, les cotés font 1.618 fois la base-  noseWIdth = distance(FKP32, FKP36)-  noseLeft = distance(FKP28, FKP32)-  noseLeft = distance(FKP28, FKP36)-  Left Nose-  ratioLeftBaseNoise = noseLeft / noseWIdth-  leftNBaseNoiseAccuracy (taux d'exactitude) = ratioLeftBaseNoise / 1.618 = 1.-  Right Nose-  ratioRightBaseNoise = noseRight / noseWIdth-  rightNBaseNoiseAccuracy (taux d'exactitude) = ratioRightBaseNoise / 1.618 = 1.>  FacialBeautyAccuracy = Sum(RuleAccuracies) / rulesNomber >  2. Train an Unsupervised classifier-  KNN (3 classes)- Note : cela peut etre utilisé comme un algorithme d'etiquetage>  3. Train a Supervised Classifier-  KNN (3 classes) as labeling algorithm-  Full Connected Neural Network as classifier 4. Deploying model as an API  Facial Keypoint Detection  This project will be all about defining and training a convolutional neural network to perform facial keypoint detection, and using computer vision techniques to transform images of faces.  The first step in any challenge like this will be to load and visualize the data you'll be working with. Let's take a look at some examples of images and corresponding facial keypoints.Facial keypoints (also called facial landmarks) are the small magenta dots shown on each of the faces in the image above. In each training and test image, there is a single face and **68 keypoints, with coordinates (x, y), for that face**.  These keypoints mark important areas of the face: the eyes, corners of the mouth, the nose, etc. These keypoints are relevant for a variety of tasks, such as face filters, emotion recognition, pose recognition, and so on. Here they are, numbered, and you can see that specific ranges of points match different portions of the face.---  Load and Visualize DataThe first step in working with any dataset is to become familiar with your data; you'll need to load in the images of faces and their keypoints and visualize them! This set of image data has been extracted from the [YouTube Faces Dataset](https://www.cs.tau.ac.il/~wolf/ytfaces/), which includes videos of people in YouTube videos. These videos have been fed through some processing steps and turned into sets of image frames containing one face and the associated keypoints. Training and Testing DataThis facial keypoints dataset consists of 5770 color images. All of these images are separated into either a training or a test set of data.* 3462 of these images are training images, for you to use as you create a model to predict keypoints.* 2308 are test images, which will be used to test the accuracy of your model.The information about the images and keypoints in this dataset are summarized in CSV files, which we can read in using `pandas`. Let's read the training CSV and get the annotations in an (N, 2) array where N is the number of keypoints and 2 is the dimension of the keypoint coordinates (x, y).---<jupyter_code># import the required libraries
import glob
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

import cv2
key_pts_frame = pd.read_csv('data/training_frames_keypoints.csv')

key_pts_frame.head(5)
n = 0
image_name = key_pts_frame.iloc[n, 0]
key_pts = key_pts_frame.iloc[n, 1:]#.as_matrix()
print(key_pts)
key_pts = key_pts.values
print(type(key_pts))
key_pts = key_pts.astype('float').reshape(-1, 2)

print('Image name: ', image_name)
print('Landmarks shape: ', key_pts.shape)
print('First 4 key pts:\n{}'.format(key_pts[:4]))
# print out some stats about the data
print('Number of images: ', key_pts_frame.shape[0])<jupyter_output><empty_output><jupyter_text>Look at some imagesBelow, is a function `show_keypoints` that takes in an image and keypoints and displays them.  As you look at this data, **note that these images are not all of the same size**, and neither are the faces! To eventually train a neural network on these images, we'll need to standardize their shape.<jupyter_code>def show_keypoints(image, key_pts):
    """Show image with keypoints"""
    plt.imshow(image)
    plt.scatter(key_pts[:, 0], key_pts[:, 1], s=20, marker='.', c='m')

# Display a few different types of images by changing the index n

# select an image by index in our data frame
n = 0
image_name = key_pts_frame.iloc[n, 0]
key_pts = key_pts_frame.iloc[n, 1:].values#.as_matrix()
key_pts = key_pts.astype('float').reshape(-1, 2)

plt.figure(figsize=(5, 5))
show_keypoints(mpimg.imread(os.path.join('data/training/', image_name)), key_pts)
plt.show()<jupyter_output><empty_output><jupyter_text>Dataset class and TransformationsTo prepare our data for training, we'll be using PyTorch's Dataset class. Much of this this code is a modified version of what can be found in the [PyTorch data loading tutorial](http://pytorch.org/tutorials/beginner/data_loading_tutorial.html). Dataset class``torch.utils.data.Dataset`` is an abstract class representing adataset. This class will allow us to load batches of image/keypoint data, and uniformly apply transformations to our data, such as rescaling and normalizing images for training a neural network.Your custom dataset should inherit ``Dataset`` and override the followingmethods:-  ``__len__`` so that ``len(dataset)`` returns the size of the dataset.-  ``__getitem__`` to support the indexing such that ``dataset[i]`` can   be used to get the i-th sample of image/keypoint data.Let's create a dataset class for our face keypoints dataset. We willread the CSV file in ``__init__`` but leave the reading of images to``__getitem__``. This is memory efficient because all the images are notstored in the memory at once but read as required.A sample of our dataset will be a dictionary``{'image': image, 'keypoints': key_pts}``. Our dataset will take anoptional argument ``transform`` so that any required processing can beapplied on the sample. We will see the usefulness of ``transform`` in thenext section.<jupyter_code>from torch.utils.data import Dataset, DataLoader

class FacialKeypointsDataset(Dataset):
    """Face Landmarks dataset."""

    def __init__(self, csv_file, root_dir, transform=None):
        """
        Args:
            csv_file (string): Path to the csv file with annotations.
            root_dir (string): Directory with all the images.
            transform (callable, optional): Optional transform to be applied
                on a sample.
        """
        self.key_pts_frame = pd.read_csv(csv_file)
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.key_pts_frame)

    def __getitem__(self, idx):
        image_name = os.path.join(self.root_dir,
                                self.key_pts_frame.iloc[idx, 0])
        
        # image matrix
        image = mpimg.imread(image_name)
        
        # if image has an alpha color channel, get rid of it
        if(image.shape[2] == 4):
            image = image[:,:,0:3]
        
        key_pts = self.key_pts_frame.iloc[idx, 1:].values#.as_matrix()

        key_pts = key_pts.astype('float').reshape(-1, 2)
        sample = {'image': image, 'keypoints': key_pts}

        if self.transform:
            sample = self.transform(sample)

        return sample<jupyter_output><empty_output><jupyter_text>Now that we've defined this class, let's instantiate the dataset and display some images.<jupyter_code># Construct the dataset
face_dataset = FacialKeypointsDataset(csv_file='data/training_frames_keypoints.csv',
                                      root_dir='data/training/')

# print some stats about the dataset
print('Length of dataset: ', len(face_dataset))
# Display a few of the images from the dataset
num_to_display = 3

for i in range(num_to_display):
    
    # define the size of images
    fig = plt.figure(figsize=(20,10))
    
    # randomly select a sample
    rand_i = np.random.randint(0, len(face_dataset))
    sample = face_dataset[rand_i]

    # print the shape of the image and keypoints
    print(i, sample['image'].shape, sample['keypoints'].shape)

    ax = plt.subplot(1, num_to_display, i + 1)
    ax.set_title('Sample #{}'.format(i))
    
    # Using the same display function, defined earlier
    show_keypoints(sample['image'], sample['keypoints'])<jupyter_output><empty_output><jupyter_text>TransformsNow, the images above are not of the same size, and neural networks often expect images that are standardized; a fixed size, with a normalized range for color ranges and coordinates, and (for PyTorch) converted from numpy lists and arrays to Tensors.Therefore, we will need to write some pre-processing code.Let's create four transforms:-  ``Normalize``: to convert a color image to grayscale values with a range of [0,1] and normalize the keypoints to be in a range of about [-1, 1]-  ``Rescale``: to rescale an image to a desired size.-  ``RandomCrop``: to crop an image randomly.-  ``ToTensor``: to convert numpy images to torch images.We will write them as callable classes instead of simple functions sothat parameters of the transform need not be passed everytime it'scalled. For this, we just need to implement ``__call__`` method and (if we require parameters to be passed in), the ``__init__`` method. We can then use a transform like this:    tx = Transform(params)    transformed_sample = tx(sample)Observe below how these transforms are generally applied to both the image and its keypoints.<jupyter_code>import torch
from torchvision import transforms, utils
# tranforms

class Normalize(object):
    """Convert a color image to grayscale and normalize the color range to [0,1]."""        

    def __call__(self, sample):
        image, key_pts = sample['image'], sample['keypoints']
        
        image_copy = np.copy(image)
        key_pts_copy = np.copy(key_pts)

        # convert image to grayscale
        image_copy = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
        
        # scale color range from [0, 255] to [0, 1]
        image_copy=  image_copy/255.0
        
        # scale keypoints to be centered around 0 with a range of [-1, 1]
        # mean = 100, sqrt = 50, so, pts should be (pts - 100)/50
        key_pts_copy = (key_pts_copy - 100)/50.0


        return {'image': image_copy, 'keypoints': key_pts_copy}


class Rescale(object):
    """Rescale the image in a sample to a given size.

    Args:
        output_size (tuple or int): Desired output size. If tuple, output is
            matched to output_size. If int, smaller of image edges is matched
            to output_size keeping aspect ratio the same.
    """

    def __init__(self, output_size):
        # output_size est de type int ou tuple
        assert isinstance(output_size, (int, tuple))
        self.output_size = output_size

    def __call__(self, sample):
        image, key_pts = sample['image'], sample['keypoints']

        h, w = image.shape[:2]
        if isinstance(self.output_size, int):
            if h > w:
                new_h, new_w = self.output_size * h / w, self.output_size
            else:
                new_h, new_w = self.output_size, self.output_size * w / h
        else:
            new_h, new_w = self.output_size # (h, w)

        new_h, new_w = int(new_h), int(new_w)

        img = cv2.resize(image, (new_w, new_h))
        
        # scale the pts, too
        key_pts = key_pts * [new_w / w, new_h / h]

        return {'image': img, 'keypoints': key_pts}


class RandomCrop(object):
    """Crop randomly the image in a sample.

    Args:
        output_size (tuple or int): Desired output size. If int, square crop
            is made.
    """

    def __init__(self, output_size):
        assert isinstance(output_size, (int, tuple))
        if isinstance(output_size, int):
            self.output_size = (output_size, output_size)
        else:
            assert len(output_size) == 2
            self.output_size = output_size

    def __call__(self, sample):
        image, key_pts = sample['image'], sample['keypoints']

        h, w = image.shape[:2]
        new_h, new_w = self.output_size

        top = np.random.randint(0, h - new_h)
        left = np.random.randint(0, w - new_w)

        image = image[top: top + new_h,
                      left: left + new_w]

        key_pts = key_pts - [left, top]

        return {'image': image, 'keypoints': key_pts}


class ToTensor(object):
    """Convert ndarrays in sample to Tensors."""

    def __call__(self, sample):
        image, key_pts = sample['image'], sample['keypoints']
         
        # if image has no grayscale color channel, add one
        if(len(image.shape) == 2):
            # add that third color dim
            image = image.reshape(image.shape[0], image.shape[1], 1)
            
        # swap color axis because
        # numpy image: H x W x C
        # torch image: C X H X W
        image = image.transpose((2, 0, 1))
        
        return {'image': torch.from_numpy(image),
                'keypoints': torch.from_numpy(key_pts)}

class AddBatchChannel(object):
    """Convert ndarrays in sample to Tensors."""

    def __call__(self, sample):
        image, key_pts = sample['image'], sample['keypoints']
         
        # if image has no grayscale color channel, add one
        if(len(image.shape) == 2):
            # add that third color dim
            image = image.reshape(image.shape[0], image.shape[1], 1)
            
        # swap color axis because
        # numpy image: H x W x C
        # batch image: C X H X W
        # image = image.transpose((2, 0, 1))
        
        return {'image': image,
                'keypoints': key_pts}<jupyter_output><empty_output><jupyter_text>Transform Compose<jupyter_code>class TransformCompose(object):
    def __init__(self, transforms=[]):
        # s'assurer qu'on a la liste de transformation
        assert isinstance(transforms, list)

        self.transforms = transforms
    
    def __call__(self, sample):
        """
            Applique la combinaison de transformations

            :return: tuple (np.array, (x, y))
        """
        # nouvelle transformation
        newSample = sample

        # appliquer les transformations
        for transform in self.transforms:
            newSample = transform(newSample)

        # add chanel
        image = newSample['image']
         
        # if image has no grayscale color channel, add one
        if(len(image.shape) == 2):
            # add that third color dim
            image = image.reshape(image.shape[0], image.shape[1], 1)
        
        # swap color axis because
        # numpy image: H x W x C
        # image: C X H X W
        image = image.transpose((2, 0, 1))

        newSample['image'] = image
        
        return newSample['image'], newSample['keypoints']<jupyter_output><empty_output><jupyter_text>Test out the transformsLet's test these transforms out to make sure they behave as expected. As you look at each transform, note that, in this case, **order does matter**. For example, you cannot crop a image using a value smaller than the original image (and the orginal images vary in size!), but, if you first rescale the original image, you can then crop it to any size smaller than the rescaled size.<jupyter_code># test out some of these transforms
rescale = Rescale(100)
crop = RandomCrop(50)
composed = transforms.Compose([Rescale(250),
                               RandomCrop(224)])

# apply the transforms to a sample image
test_num = 500
sample = face_dataset[test_num]

fig = plt.figure()
for i, tx in enumerate([rescale, crop, composed]):
    transformed_sample = tx(sample)

    ax = plt.subplot(1, 3, i + 1)
    plt.tight_layout()
    ax.set_title(type(tx).__name__)
    show_keypoints(transformed_sample['image'], transformed_sample['keypoints'])

plt.show()<jupyter_output><empty_output><jupyter_text>Create the transformed datasetApply the transforms in order to get grayscale images of the same shape. Verify that your transform works by printing out the shape of the resulting data (printing out a few examples should show you a consistent tensor size).<jupyter_code># define the data tranform
# order matters! i.e. rescaling should come before a smaller crop
data_transform = transforms.Compose([Rescale(250),
                                     RandomCrop(224),
                                     Normalize(),
                                     ToTensor()])

# create the transformed dataset
transformed_dataset = FacialKeypointsDataset(csv_file='data/training_frames_keypoints.csv',
                                             root_dir='data/training/',
                                             transform=data_transform)

# print some stats about the transformed data
print('Number of images: ', len(transformed_dataset))

# make sure the sample tensors are the expected size
for i in range(5):
    sample = transformed_dataset[i]
    print(i, sample['image'].size(), sample['keypoints'].size())<jupyter_output><empty_output><jupyter_text>Data Iteration and BatchingRight now, we are iterating over this data using a ``for`` loop, but we are missing out on a lot of PyTorch's dataset capabilities, specifically the abilities to:-  Batch the data-  Shuffle the data-  Load the data in parallel using ``multiprocessing`` workers.``torch.utils.data.DataLoader`` is an iterator which provides all thesefeatures, and we'll see this in use in the *next* notebook, Notebook 2, when we load data in batches to train a neural network!---<jupyter_code>BATCH_SIZE          = 128
IMG_SHAPE  = 96  # Our training data consists of images with width of 150 pixels and height of 150 pixels<jupyter_output><empty_output><jupyter_text>Dataset with no any transformation<jupyter_code># create the transformed training dataset

# data_transform = transforms.Compose([Rescale(150),
#                                      RandomCrop(IMG_SHAPE),
#                                      Normalize(),
#                                      AddBatchChannel()])

data_transform = None

trainingDataset = FacialKeypointsDataset(csv_file='data/training_frames_keypoints.csv',
                                             root_dir='data/training/',
                                             transform=data_transform)

# create the transformed testing dataset
# data_transform = transforms.Compose([Rescale((IMG_SHAPE, IMG_SHAPE)),
#                                      Normalize(),
#                                      AddBatchChannel()])

data_transform = None


testingDataset = FacialKeypointsDataset(csv_file='data/test_frames_keypoints.csv',
                                             root_dir='data/test/',
                                             transform=data_transform)<jupyter_output><empty_output><jupyter_text>Dataset with transformation<jupyter_code># create the transformed training dataset
data_transform = transforms.Compose([Rescale(150),
                                     RandomCrop(IMG_SHAPE),
                                     Normalize(),
                                     AddBatchChannel()])



trainingDataset = FacialKeypointsDataset(csv_file='data/training_frames_keypoints.csv',
                                             root_dir='data/training/',
                                             transform=data_transform)


# create the transformed testing dataset
data_transform = transforms.Compose([Rescale((IMG_SHAPE, IMG_SHAPE)),
                                     Normalize(),
                                     AddBatchChannel()])



testingDataset = FacialKeypointsDataset(csv_file='data/test_frames_keypoints.csv',
                                             root_dir='data/test/',
                                             transform=data_transform)                                         
sample = trainingDataset[0]

print(f"image shape : {sample['image'].shape} and FKPs shape : {sample['keypoints'].shape}")
sample = testingDataset[0]

print(f"image shape : {sample['image'].shape} and FKPs shape : {sample['keypoints'].shape}")<jupyter_output><empty_output><jupyter_text>Train data and Validation data>  - Train data : 80% train dataset>  - Validation data : 20% train dataset  Transform dataset to numpy inputs and targets<jupyter_code>def transformDataset(dataset):
    inputs  = []
    targets = []

    for sample in dataset:
        image       = sample['image']
        keypoints   = sample['keypoints']

        image       = np.array(image)
        keypoints   = np.array(keypoints)

        inputs.append(image)
        targets.append(keypoints)
    
    inputs  = np.array(inputs)
    targets = np.array(targets)

    return inputs, targets<jupyter_output><empty_output><jupyter_text>Transformation example<jupyter_code># separation de donnees
from sklearn.model_selection import train_test_split
import numpy as np
VALIDATION_SPLIT    = 0.20<jupyter_output><empty_output><jupyter_text>Fake data for testing<jupyter_code>nb = 3000
inputs = np.random.rand(nb,  96, 96, 1)
targets = np.random.rand(nb,68,2)

print(f"inputs.shape : {inputs.shape}\ttargets.shape : {targets.shape}")<jupyter_output>inputs.shape : (3000, 96, 96, 1)	targets.shape : (3000, 68, 2)<jupyter_text>Dataset Transformation<jupyter_code>inputs, targets = transformDataset(trainingDataset)

print(f"inputs.shape : {inputs.shape}\ttargets.shape : {targets.shape}")

inputs[0].shape
inputs[1].shape<jupyter_output><empty_output><jupyter_text>Split dataset into training and validation<jupyter_code>trainDatas, validationDatas, trainLabels, validationLabels = train_test_split(
    inputs,targets, 
    test_size = VALIDATION_SPLIT, # 0.2
    shuffle = True
)

# Training datas
trainDatas = np.array(trainDatas)
trainLabels = np.array(trainLabels)

# Validation datas
validationDatas = np.array(validationDatas)
validationLabels = np.array(validationLabels)

trainDatas.shape
validationDatas.shape<jupyter_output><empty_output><jupyter_text>Testing data transformation<jupyter_code># Testing datas
testDatas, testLabels = transformDataset(testingDataset)

testDatas = np.array(testDatas)
testLabels = np.array(testLabels)

print(f"inputs.shape : {testDatas.shape}\ttargets.shape : {testLabels.shape}")
testDatas.shape<jupyter_output><empty_output><jupyter_text>Custom generator for memory optimization<jupyter_code>class DataGenerator(object):

    # class variables
    TRAINING = "training"
    TESTING = "testing"
    VALIDATION = "validation"


    def __init__(self, targetLabel=0, targetData='training', batchSize=20):
        # targetLabel index must be int
        assert isinstance(targetLabel, int)

        # targetLata must be str 
        assert isinstance(targetData, str) and targetData in [self.TRAINING, self.TESTING, self.VALIDATION]

        # batchSize index must be int
        assert isinstance(batchSize, int)

        self.targetLabel = targetLabel
        self.targetData = targetData
        self.batchSize = batchSize

        # counter 
        self.lastbatch = 0


        # compute size
        self.size = self.shape()[0]       

    def __len__(self):
        return self.size

    def __iter__(self):
        return self
    
    def __next__(self):
        return self.next()
    
    def __repr__(self):
        return (
            f"DataGenerator\n\n"
            f"targetData  : {self.targetData}\n"
            f"targetLabel : {self.targetLabel}\n"
            f"size        : {self.size}\n"
            f"batchSize   : {self.batchSize}\n"

        )
    
    def shape(self):
        if self.targetData == self.TRAINING:
            return trainDatas.shape

        elif self.targetData == self.TESTING:
            return testDatas.shape

        elif self.targetData == self.VALIDATION:
            return validationDatas.shape
        
    def makeBatch(self, datas, targets, selectedIndex):
        """
            Make a batch
        """

        batch = []

        for index in range(self.lastbatch, selectedIndex):
            # index out of range
            if index >= self.size:
                break

            # sample, image and target FPK
            sample = (datas[index], targets[index][self.targetLabel])

            batch.append(sample)
        
        return batch

    
    def next(self):
        # index out of size
        if self.lastbatch >= self.size:
            self.lastbatch = 0
            raise StopIteration()

        # batch size
        batch = list()

        # last index
        lastindex = self.lastbatch + self.batchSize

        if self.targetData == self.TRAINING:
            batch = self.makeBatch(trainDatas, trainLabels, lastindex)

        elif self.targetData == self.TESTING:
            batch = self.makeBatch(testDatas, testLabels, lastindex)

        elif self.targetData == self.VALIDATION:
            batch = self.makeBatch(validationDatas, validationLabels, lastindex)

        # next batch
        self.lastbatch = lastindex

        if len(batch) == 0:
            self.lastbatch = 0
            raise StopIteration()

        return batch
        

trainingGenTest = DataGenerator(targetData='training', targetLabel=0, batchSize=20)
trainingGenTest

for data in trainingGenTest:
    print(f"Data Size  : {len(data)}\t- Data Type : {type(data)}\n")

    sample = data[0]

    print(f"Image size : {sample[0].shape}\t- Target Size : {sample[1].shape}\n")

    break<jupyter_output><empty_output><jupyter_text>Defining and Training a Convolutional Neural Network (CNN) to Predict Facial Keypoints<jupyter_code>try:
  # Use the %tensorflow_version magic if in colab.
  %tensorflow_version 2.x
except Exception:
  pass

import tensorflow as tf
naimishModel = tf.keras.models.Sequential([
    # First layer
    # Convolution2d1 to Convolution2d4 do not use zero
    # padding, have their weights initialized with random numbers drawn from uniform distribution
    tf.keras.layers.Conv2D(32, (4,4), input_shape=(96, 96, 1), padding='valid', activation='elu',
        kernel_initializer=tf.keras.initializers.RandomUniform(minval=-1., maxval=1.)
    ),
    
    # Maxpooling2d1 to Maxpooling2d4 use a pool shape of (2, 2), with non-overlapping strides and no zero padding
    tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=None, padding="valid"),
    tf.keras.layers.Dropout(0.1),

    # second layer
    tf.keras.layers.Conv2D(64, (3,3), padding='valid', activation='elu',
        kernel_initializer=tf.keras.initializers.RandomUniform(minval=-1., maxval=1.)
    ),
    tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=None, padding="valid"),
    tf.keras.layers.Dropout(0.2),

    # Third layer
    tf.keras.layers.Conv2D(128, (2,2), padding='valid', activation='elu',
        kernel_initializer=tf.keras.initializers.RandomUniform(minval=-1., maxval=1.)
    ),
    tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=None, padding="valid"),
    tf.keras.layers.Dropout(0.3),

    # Fourth layer
    tf.keras.layers.Conv2D(256, (1,1), padding='valid', activation='elu',
        kernel_initializer=tf.keras.initializers.RandomUniform(minval=-1., maxval=1.)
    ),
    tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=None, padding="valid"),
    tf.keras.layers.Dropout(0.4),

    # Flatten
    tf.keras.layers.Flatten(),

    # Dense 1
    tf.keras.layers.Dense(1000, activation='elu'),
    tf.keras.layers.Dropout(0.5),

    # Dense 2
    tf.keras.layers.Dense(1000, activation='elu'),
    tf.keras.layers.Dropout(0.6),

    # Dense 1
    tf.keras.layers.Dense(2, activation='elu'),

])<jupyter_output>WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\keras\initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:Large dropout rate: 0.6 (>0.5). In TensorFlow 2.x, dropout() uses dropout rate instead of keep_prob. Please ensure that this is intended.<jupyter_text>Learning Configuration<jupyter_code>optimizer = tf.keras.optimizers.Adam(
    learning_rate=0.001,
    beta_1=0.9,
    beta_2=0.999,
    epsilon=1e-08,
    amsgrad=False,
    name="Adam"
)

naimishModel.compile(
    optimizer=optimizer,
    loss=tf.keras.metrics.mean_squared_error,
    metrics=['accuracy']
)<jupyter_output><empty_output><jupyter_text>Model Summary<jupyter_code>naimishModel.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 93, 93, 32)        544       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 46, 46, 32)        0         
_________________________________________________________________
dropout (Dropout)            (None, 46, 46, 32)        0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 44, 44, 64)        18496     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 22, 22, 64)        0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 22, 22, 64)        0         
________________________________________________________[...]<jupyter_text>Test clone keras model<jupyter_code>modelClone = tf.keras.models.clone_model(naimishModel)

print(f"\nmodelClone == naimishModel : {modelClone == naimishModel}\n")
print(f"modelClone is naimishModel : {modelClone is naimishModel}\n")
print(f"\nid(modelClone)   : {id(modelClone)}\n\nid(naimishModel) : {id(naimishModel)}")
modelClone.summary()<jupyter_output><empty_output><jupyter_text>Training>  - Train data : 80% train dataset>  - Validation data : 20% train dataset>  - Batch SIze : 128>  - Epochs : 300>  - Early Stopping Callback (ESC) :  stops the training when the number of contiguous epochs without improvement in validation loss are 30 (Patience Level . >  -  Model Checkpoint Callback (MCC) : stores the weights of the model with the lowest validation loss.  Callback functions<jupyter_code>BATCH_SIZE          = 128
EPOCHS              = 300
PATIENCE            = 30
import os

from datetime import datetime


class CustomCallback(tf.keras.callbacks.Callback):

    def __init__(self, cwd, targetFolder):

        assert os.path.exists(cwd)
        assert isinstance(targetFolder, str)
        assert os.path.exists(targetFolder)

        self.targetFolder = targetFolder
        # self.modelRef = modelRef
        self.cwd = cwd

    def on_epoch_end(self, epoch, logs=None):
        keys = list(logs.keys())
        print("\nEnd epoch {} of training; got log keys: {}".format(epoch, keys))

        # metrics
        valLoss = logs['val_loss']

        if 'val_acc' in logs:
            valAcc = logs['val_acc']
        
        elif 'val_accuracy' in logs:
            valAcc = logs['val_accuracy']

        try:
            # swap cwd
            os.chdir(self.targetFolder)
            # save model
            modelFile = f"model-{datetime.now().timestamp()}-{epoch}-{valLoss:.3f}-{valAcc:.3f}.h5"
            
            print("\nSaving model : '{}'\n".format(modelFile))


            self.model.save(modelFile)

        except Exception as error:
            print(error)

        finally:
            # swap cwd
            os.chdir(self.cwd)




import os

modelClone = tf.keras.models.clone_model(naimishModel)

cwd = os.getcwd()

logDir = os.path.join(cwd, 'modelTest', 'logs')
modelTest = os.path.join(cwd, 'modelTest')


if not os.path.isdir(logDir):
    os.makedirs(logDir)

modelFile = 'model.{epoch:02d}-{val_loss:.2f}.h5'

my_callbacks = [
    tf.keras.callbacks.EarlyStopping(patience=PATIENCE),
    # tf.keras.callbacks.ModelCheckpoint(filepath=modelFile),
    tf.keras.callbacks.TensorBoard(log_dir=logDir),
    CustomCallback(cwd=cwd, targetFolder=modelTest)
]
# history = modelClone.fit(
#     trainDatas,
#     trainLabels[:, 0, :],
#     epochs=1,
#     validation_data = (validationDatas, validationLabels[:, 0, :]),
#     callbacks=my_callbacks,
#     verbose=1

# )<jupyter_output><empty_output><jupyter_text>Toujours compiler un model apres clonage You must compile your model before training/testing. Use `model.compile(optimizer, loss)`.<jupyter_code>modelClone.compile(
    optimizer=optimizer,
    loss=tf.keras.metrics.mean_squared_error,
    metrics=['accuracy']
)

history = modelClone.fit(
    trainDatas,
    trainLabels[:, 0, :],
    epochs=1,
    validation_data = (validationDatas, validationLabels[:, 0, :]),
    callbacks=my_callbacks,
    verbose=1
)<jupyter_output><empty_output><jupyter_text>Global and parallel training<jupyter_code>import os
import shutil
def makeFolder(targetPath):
    if not os.path.isdir(targetPath):
        os.makedirs(targetPath)
    

def removeFolder(targetPath):
    if os.path.isdir(targetPath):
        shutil.rmtree(targetPath, ignore_errors=True)<jupyter_output><empty_output><jupyter_text>Save history object Helper<jupyter_code>import pickle

class HistoryManager(object):
    def __init__(self, targetFile, data={}):
        assert isinstance(targetFile, str)

        self.filename = f"{targetFile}.data"

        self.data = data
    
    def save(self, targetObject=None):
        if targetObject != None:
            self.data = targetObject

        with open(self.filename, 'wb') as handle:
            pickle.dump(self.data, handle, protocol=pickle.HIGHEST_PROTOCOL)
    
    def load(self):
        with open(self.filename, 'rb') as handle:
            data = pickle.load(handle)
        
        self.data = data
        return data

a = {"a": "b"}

am = HistoryManager("Jonathan", a)

am.save()
b = am.load()
b<jupyter_output><empty_output><jupyter_text>Train model Helper<jupyter_code>trainLabels.shape
trainLabels[:, 0, :].shape
from threading import RLock


def trainModel(model, fkp, my_callbacks, epochs, fkpFolder):

    # lock = RLock()

    # with lock:
    # fit model
    history = model.fit(
        trainDatas,
        trainLabels[:, fkp, :],
        epochs=epochs,
        validation_data = (validationDatas, validationLabels[:, fkp, :]),
        callbacks=my_callbacks,
        verbose=1
    )

    # save history
    historyFile = os.path.join(fkpFolder, 'history')

    history = HistoryManager(historyFile, history)
    history.save()<jupyter_output><empty_output><jupyter_text>Make training Thread<jupyter_code>import os
from threading import Thread


# current work directory
cwd = os.getcwd()
cwd
import os
from threading import Thread


# current work directory
cwd = os.getcwd()

# models folder
modelsFolder = os.path.join(cwd, 'models')

# remove folder
removeFolder(modelsFolder)

makeFolder(modelsFolder)


# Thread list
trainingThreads = []

# targetFKPs : index begin by 0
targetFKPs = np.array([49, 55, 28, 32, 36, 4, 14]) - 1

# iter FKPS
for fkp in targetFKPs:
    # clone model
    modelClone = tf.keras.models.clone_model(naimishModel)

    # compile clone model
    modelClone.compile(
        optimizer=optimizer,
        loss=tf.keras.metrics.mean_squared_error,
        metrics=['accuracy']
    )

    # define callbacks
    fkpFolder = os.path.join(modelsFolder, str(f"FKP{fkp}"))
    makeFolder(fkpFolder)


    logDir = os.path.join(fkpFolder, 'logs')
    makeFolder(logDir)
    
    modelFile = 'model.{epoch:02d}-{val_loss:.2f}.h5'

    my_callbacks = [
        tf.keras.callbacks.EarlyStopping(patience=PATIENCE),
        # tf.keras.callbacks.ModelCheckpoint(filepath=modelFile),
        tf.keras.callbacks.TensorBoard(log_dir=logDir),
        CustomCallback(cwd=cwd, targetFolder=fkpFolder)

    ]

    # create training thread
    # def trainModel(model, fkp, my_callbacks, epochs, fkpFolder):

    # trainThread = Thread(
    #     target=trainModel,
    #     args=(modelClone, fkp, my_callbacks, 1, fkpFolder),
    #     name=f"TrainingTread#{fkp}"
    # )

    # trainModel(modelClone, fkp, my_callbacks, 1, fkpFolder)

    history = modelClone.fit(
        trainDatas,
        trainLabels[:, fkp, :],
        validation_data = (validationDatas, validationLabels[:, fkp, :]),
        epochs=EPOCHS,
        batch_size=BATCH_SIZE,
        callbacks=my_callbacks,
        verbose=1
    )

    # save history
    historyFile = os.path.join(fkpFolder, 'history')
    history = HistoryManager(historyFile, history.history)
    history.save()
    
    # add thread to training tread list
    # trainingThreads.append(trainThread)<jupyter_output><empty_output><jupyter_text>PUsh Result and shutdown VM<jupyter_code>! git add *
! git commit -m "Training result"
! git push origin training
! shutdown -s -t 100<jupyter_output><empty_output><jupyter_text>Result Visualization<jupyter_code>acc = history.history['accuracy']
val_acc = history.history['val_accuracy']

loss = history.history['loss']
val_loss = history.history['val_loss']

epochs_range = range(epochs)

plt.figure(figsize=(8, 8))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()<jupyter_output><empty_output><jupyter_text>Export model<jupyter_code>import time
import os

savedModelPath = os.path.join(os.getcwd(), f"saved_models/")

if not os.path.exists(savedModelPath):
  os.makedirs(savedModelPath)

model.trainable = False
model.save(savedModelPath)<jupyter_output><empty_output><jupyter_text>Load saved model<jupyter_code>try:
  # Use the %tensorflow_version magic if in colab.
  %tensorflow_version 2.x
except Exception:
  pass

import tensorflow as tf<jupyter_output><empty_output><jupyter_text>Load Model Helper<jupyter_code>import os


cwd = os.getcwd()


def loadFKPModel(fkp, name):
    targetFile = os.path.join(cwd, "models", f"FKP{fkp}", name)

    # assert that this model file exists
    assert os.path.exists(targetFile)

    # load model
    loadedModel = tf.keras.models.load_model(targetFile)

    return loadedModel

saved_model_path = "D:\FacialBeauty-AI\models\FKP0\model.01-1.80.h5"

loadedModel = loadFKPModel(fkp=0, name="model.01-3.24.h5")

loadModel.summary()<jupyter_output>WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
[...]<jupyter_text>Load History Helper<jupyter_code>import os


cwd = os.getcwd()


def loadModelHistory(fkp):
    targetFile = os.path.join(cwd, "models", f"FKP{fkp}", "history")

    history = HistoryManager(targetFile=targetFile)

    return history.data

history = loadModelHistory(fkp=0)
history<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1006021962451293, 'fraction_numerical': 0.014287401109930334, 'mean_word_length': 3.2303696303696303, 'pattern_counts': {'":': 1, '<': 133, '<?xml version=': 0, '>': 174, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 37, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '34', 'dir': 'jupyter-structured-clean-dedup', 'id': '512690', 'license': 'MIT', 'n_tokens_mistral': 13425, 'n_tokens_neox': 12556, 'n_words': 3653, 'path': 'FacialBeauty-AI.ipynb', 'repo_name': 'nathanbangwa243/FacialBeauty-AI', 'size': '42345'}