<jupyter_start><jupyter_text>Deep Learning Applied to Images - Feature Extraction Prof. <NAME>  Feature extraction example* Using Keras* Using the convolutional neural network Inception-v3  Imports<jupyter_code>import os
import numpy as np
import matplotlib.pyplot as plt

from pprint import pprint
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.inception_v3 import InceptionV3
from tensorflow.keras.applications.inception_v3 import preprocess_input

plt.rcParams['figure.figsize'] = [15, 5]<jupyter_output><empty_output><jupyter_text>Creating the model ```python Use model pretrained on ImageNetweights='imagenet' Use the network without the final classification layerinclude_top=False Indicated we would like to have an average pooling in the last layerspooling='avg' Input shape. In this case, the image has 299px of height, 299px of width, and 3 channels.input_shape=(299, 299, 3)```<jupyter_code>model = InceptionV3(weights='imagenet', include_top=False, pooling='avg', input_shape=(299, 299, 3))
model.summary()<jupyter_output>Model: "inception_v3"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 299, 299, 3) 0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 149, 149, 32) 864         input_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 149, 149, 32) 96          conv2d[0][0]                     
__________________________________________________________________________________________________
activation (Activation)         (None, 149, 149, 32) 0           batch_normalization[0][...]<jupyter_text>Feature extraction example<jupyter_code>img_path = 'sample_images/sneakers.png'
img = image.load_img(img_path, target_size=(299, 299))
plt.imshow(img)<jupyter_output><empty_output><jupyter_text>We need to prepare the image using the same preprocessing steps used to train the model. Fortunetly, Keras has methods to help us out.<jupyter_code>x = image.img_to_array(img)
print(x.shape, x.dtype)
x = np.expand_dims(x, axis=0)
print(x.shape, x.dtype)
x = preprocess_input(x)

features = model.predict(x)<jupyter_output>(299, 299, 3) float32
(1, 299, 299, 3) float32<jupyter_text>Checking shape and type<jupyter_code>print(features.shape, features.dtype)<jupyter_output>(1, 2048) float32<jupyter_text>Printing features<jupyter_code>pprint(features)<jupyter_output>array([[0.17176437, 0.29913104, 0.5031217 , ..., 0.11340574, 0.04080743,
        0.38091427]], dtype=float32)<jupyter_text>Using features for comparisonImage's features can be used for comparison. For example, we can compute the feautres of an image and compare with the features of other images, with the goal of retrieving the most similar ones.<jupyter_code>def load_and_extract_features(model, img_path):
    # Loading RGBA to show the image properly
    img = image.load_img(img_path, color_mode='rgba')
    plt.imshow(img)
    
    # Loading RGB with expected input size
    img = image.load_img(img_path, target_size=(299, 299))
    
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)

    features = model.predict(x)
    
    return features
features_tshirt1 = load_and_extract_features(model, 'sample_images/tshirt.png')
features_tshirt2 = load_and_extract_features(model, 'sample_images/tshirt2.png')
features_pug = load_and_extract_features(model, 'sample_images/pug.png')
features_pug2 = load_and_extract_features(model, 'sample_images/pug2.png')
features_sneakers = load_and_extract_features(model, 'sample_images/sneakers.png')<jupyter_output><empty_output><jupyter_text>Computing distance between featuresWe can then compute the distance between these features and see whether given images are more similar to each other<jupyter_code>from scipy.spatial.distance import cosine<jupyter_output><empty_output><jupyter_text>T-shirt 1 vs Pug 1<jupyter_code>distance = cosine(features_tshirt1, features_pug)
print(distance)<jupyter_output>0.712799459695816<jupyter_text>T-shirt 2 vs Pug 2<jupyter_code>distance = cosine(features_tshirt2, features_pug2)
print(distance)<jupyter_output>0.6621387302875519<jupyter_text>Pug 1 vs Sneakers<jupyter_code>distance = cosine(features_pug, features_sneakers)
print(distance)
distance = np.linalg.norm(features_pug - features_sneakers)
print(distance)<jupyter_output>16.784641<jupyter_text>T-shirt 1 vs T-shirt 2<jupyter_code>distance = cosine(features_tshirt1, features_tshirt2)
print(distance)
distance = np.linalg.norm(features_tshirt1 - features_tshirt2)
print(distance)<jupyter_output>8.058087<jupyter_text>Pug 1 vs Pug 2<jupyter_code>distance = cosine(features_pug, features_pug2)
print(distance)
distance = np.linalg.norm(features_pug - features_pug2)
print(distance)
distance = cosine(features_pug, features_pug)
print(distance)<jupyter_output>0.0
{'fraction_non_alphanumeric': 0.1044776119402985, 'fraction_numerical': 0.046476478367655394, 'mean_word_length': 5.974967061923584, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '296034', 'license': 'Apache-2.0', 'n_tokens_mistral': 1806, 'n_tokens_neox': 1618, 'n_words': 367, 'path': '3.feature_extraction/3.2_feature_extraction_keras_inception.ipynb', 'repo_name': 'tspthomas/deep_learning_for_images', 'size': '5308'}