<jupyter_start><jupyter_text><jupyter_code>%tensorflow_version 2.x<jupyter_output>TensorFlow 2.x selected.<jupyter_text>In this and the activation_maximization.ipynb notebooks we will try to understand the visualization technique presented in the following paper:[Deep Inside Convolutional Networks: VisualisingImage Classification Models and Saliency Maps](https://arxiv.org/pdf/1312.6034.pdf).Here we are trying to find the regions of image which are mostly responsible for a specific class prediction NOTE:In the paper under section 3, the authors approximate the linear scoring function by using the 1st order term from the Tailor expansion of the softmax function (See equations 2 and 3). Instead of taking the 1st order term from the Tailor expansion we can do another trick to approximate a Linear scoring function: Just Swap the Softmax activation to a Linear/Identity activation in the output.<jupyter_code>import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import cv2
model = tf.keras.applications.vgg16.VGG16(include_top=True, weights="imagenet")
model.summary()
image = np.array(tf.keras.preprocessing.image.load_img(path="./images/labrador.jpeg",target_size=(224,224)))
plt.imshow(image)
plt.show()
preprocessed_image = tf.keras.applications.vgg16.preprocess_input(image.astype(np.float32))
preprocessed_image = np.expand_dims(preprocessed_image, axis=0)  # reshape it to (1,224,224,3),
# trick to get optimal visualizations swap softmax with linear
model.get_layer("predictions").activation = None
def get_gradients(model, image, class_index):
    image_tensor = tf.convert_to_tensor(image, dtype="float32")
    with tf.GradientTape() as tape:
        tape.watch(image_tensor)
        output = model(image_tensor)
        loss = tf.reduce_mean(output[:, class_index])
    grads = tape.gradient(loss, image_tensor)
    return grads
class_index = 208  # labrador
grads = get_gradients(model, preprocessed_image, class_index=class_index)
gradient_image = grads.numpy()[0]
gradient_image.shape
saliency_map = np.max(np.abs(gradient_image), axis=2)
saliency_map.shape
plt.imshow(saliency_map)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08633425669436749, 'fraction_numerical': 0.02123730378578024, 'mean_word_length': 6.104918032786885, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '506399', 'license': 'MIT', 'n_tokens_mistral': 705, 'n_tokens_neox': 651, 'n_words': 194, 'path': 'saliency_map.ipynb', 'repo_name': 'Shivvrat/Visualizing-and-Understanding-CNNs-', 'size': '2166'}