<jupyter_start><jupyter_text>Visualizing filters in Pytorch  Imports<jupyter_code>import matplotlib.pyplot as plt
from PIL import Image
import torchvision
import torch
import os
import numpy as np
%matplotlib inline
image = Image.open("images/zebra.jpg")
plt.imshow(image)
print("Image shape:", image.size)<jupyter_output>Image shape: (224, 224)<jupyter_text>Model definitionIn this example we will use a pre-trained ResNet50 network. ResNet-50 is a fully-convolutional neural network that excels at image classification.<jupyter_code>model = torchvision.models.resnet50(pretrained=True)
print(model)<jupyter_output>Downloading: "https://download.pytorch.org/models/resnet50-19c8e357.pth" to /Users/anastasia/.cache/torch/checkpoints/resnet50-19c8e357.pth
100%|██████████| 97.8M/97.8M [00:13<00:00, 7.54MB/s]<jupyter_text>First convolution layerIn this task we are interested in visualizing the first convolutional layer. This can be retrieved by the following code block:We can see that it has 64 filters/kernels in the layer. Each kernel is a $7 \times 7$ filter, that takes an RGB image as input<jupyter_code>first_conv_layer = model.conv1
print("First conv layer weight shape:", first_conv_layer.weight.shape)
print("First conv layer:", first_conv_layer)<jupyter_output>First conv layer weight shape: torch.Size([64, 3, 7, 7])
First conv layer: Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)<jupyter_text>Activation from first layerWe can retrieve the activation from the first layer by doing a forward pass throught this conv layer.However, first we need to resize, and normalize the image with the mean and standard deviation that they used to originally train this network.<jupyter_code>image_transform = torchvision.transforms.Compose([
    torchvision.transforms.Resize((224, 224)),
    torchvision.transforms.ToTensor(),
    torchvision.transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225])
])
image = image_transform(image)[None]
print("Image shape:", image.shape)
activation = first_conv_layer(image)
print("Activation shape:", activation.shape)<jupyter_output>Activation shape: torch.Size([1, 64, 112, 112])<jupyter_text>Visualize filters & Activation<jupyter_code># tip
# To transform a weight to a numpy array, you can use to function
def torch_image_to_numpy(image: torch.Tensor):
    # Normalize to [0 - 1.0]
    image = image.detach().cpu() # Transform image to CPU memory (if on GPU VRAM)
    image = image - image.min()
    image = image / image.max()
    image = image.numpy()
    if len(image.shape) == 2: # Grayscale image, can just return
        return image
    assert image.shape[0] == 3, "Expected color channel to be on first axis. Got: {}".format(image.shape)
    image = np.moveaxis(image, 0, 2)
    return image
    

### START YOUR CODE HERE ### (You can change anything inside this block)
# plt.subplot is a nice function to use for this task!
indices = [5, 8, 19, 22, 34]

plt.figure(figsize=(20, 4)) 

for index, i in enumerate(indices):
    plt.subplot(2, 5, index + 1)
    filter_im = torch_image_to_numpy(first_conv_layer.weight[i, ...])
    plt.imshow(filter_im)
    plt.subplot(2, 5, index + 5 + 1)
    activation_im = torch_image_to_numpy(activation[0, i, :, :])
    plt.imshow(activation_im, cmap="gray")
    
plt.savefig(os.path.join("plots", "task2d.png"))


### END YOUR CODE HERE ###<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11845972957084068, 'fraction_numerical': 0.03968253968253968, 'mean_word_length': 5.065953654188949, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 17, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '411722', 'license': 'MIT', 'n_tokens_mistral': 1227, 'n_tokens_neox': 1110, 'n_words': 337, 'path': 'assignment2/task2d.ipynb', 'repo_name': 'anastalind/TDT4195-StarterCode', 'size': '3402'}