<jupyter_start><jupyter_text><jupyter_code>from google.colab import drive
drive.mount('/content/drive')
%load_ext autoreload
%autoreload 2
import torch
from torch.utils.data import DataLoader
from torch.optim import Adam
from matplotlib import pyplot as plt
from utils import get_mnist_data
from models import ConvNN
from training_and_evaluation import train_model, predict_model
from attacks import fast_gradient_attack
from torch.nn.functional import cross_entropy
import os
if not os.path.isdir("models"):
    os.mkdir("models")<jupyter_output>The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload<jupyter_text>Project 2, part 1: Creating adversarial examples (50 pt)In this notebook we train a basic convolutional neural network on MNIST and craft adversarial examples via gradient descent. Your taskComplete the missing code in the respective files, i.e. `training_and_evaluation.py`, `attacks.py`, and this notebook. Make sure that all the functions follow the provided specification, i.e. the output of the function exactly matches the description in the docstring. Specifically, for this part you will have to implement the following functions / classes:  **`training_and_evaluation.py`**:* `train_model` (15pt)* `predict_model` (10pt)**`attacks.py`**:* `fast_gradient_attack` (15pt)**This notebook*** Cells in the Qualitative Evaluation section. (10pt) General remarksDo not add or modify any code outside of the following comment blocks, or where otherwise explicitly stated.``` python YOUR CODE HERE...```After you fill in all the missing code, restart the kernel and re-run all the cells in the notebook.The following things are **NOT** allowed:- Using additional `import` statements- Copying / reusing code from other sources (e.g. code by other students)If you plagiarise even for a single project task, you won't be eligible for the bonus this semester.<jupyter_code>mnist_trainset = get_mnist_data(train=True)
mnist_testset = get_mnist_data(train=False)

use_cuda = torch.cuda.is_available() #and False

model = ConvNN()
if use_cuda:
    model = model.cuda()

epochs = 1
batch_size = 128
test_batch_size = 1000  # feel free to change this
lr = 1e-3

opt = Adam(model.parameters(), lr=lr)
def loss_function(x, y, model):
    logits = model(x).cpu()
    loss = cross_entropy(logits, y)
    return loss, logits<jupyter_output><empty_output><jupyter_text>Implement the `train_model` function in the file `training_and_evaluation.py`.<jupyter_code>losses, accuracies = train_model(model, mnist_trainset, batch_size=batch_size, loss_function=loss_function, optimizer=opt)
torch.save(model.state_dict(), "models/standard_training.checkpoint")
model.load_state_dict(torch.load("models/standard_training.checkpoint", map_location="cpu"))
fig = plt.figure(figsize=(10,3))
plt.subplot(121)
plt.plot(losses)
plt.xlabel("Iteration")
plt.ylabel("Training Loss")
plt.subplot(122)
plt.plot(accuracies)
plt.xlabel("Iteration")
plt.ylabel("Training Accuracy")
plt.show()<jupyter_output><empty_output><jupyter_text>Implement the `predict_model` function in the file `training_and_evaluation.py`.<jupyter_code>clean_accuracy = predict_model(model, mnist_testset, batch_size=test_batch_size, attack_function=None)<jupyter_output><empty_output><jupyter_text>Creating adversarial examples $L_2$-bounded attacksFist, craft adversarial perturbations that have a $L_2$ norm of $ \| \tilde{\mathbf{x}} - \mathbf{x} \|_2 = \epsilon$ with $\epsilon=5$. $L_\infty$-bounded attacksAfterwards, craft adversarial perturbations with $L_\infty$ norm of $ \| \tilde{\mathbf{x}} - \mathbf{x} \|_\infty = \epsilon$ with $\epsilon=0.3$.For this you need to implement `predict_model` in the file `training_and_evaluation.py` and `fast_gradient_attack` in `attacks.py`. See the docstring comments there.<jupyter_code>attack_args_l2 = {"epsilon": 5, "norm": "2"}
attack_args_linf = {"epsilon": 0.3, "norm": "inf"}<jupyter_output><empty_output><jupyter_text>Qualitative evaluationFirst, craft adversarial examples for 10 randomly selected test samples and inspect them by plotting them. $L_2$ attack:<jupyter_code>test_loader = DataLoader(mnist_testset, batch_size = 10, shuffle=True)
x,y = next(iter(test_loader))
##########################################################
# YOUR CODE HERE
...
##########################################################<jupyter_output><empty_output><jupyter_text>$L_\infty$ attack:<jupyter_code>##########################################################
# YOUR CODE HERE
...
##########################################################<jupyter_output><empty_output><jupyter_text>Visualize the adversarial examples and the model's prediction on them:<jupyter_code>for ix in range(len(x)):
    plt.subplot(131)
    plt.imshow(x[ix,0].detach().cpu(), cmap="gray")
    plt.title(f"Label: {y[ix]}")
    plt.subplot(132)
    plt.imshow(x_pert_l2[ix,0].detach().cpu(), cmap="gray")
    plt.title(f"Predicted: {y_pert_l2[ix]}")
    
    plt.subplot(133)
    plt.imshow(x_pert_linf[ix,0].detach().cpu(), cmap="gray")
    plt.title(f"Predicted: {y_pert_linf[ix]}")
    plt.show()<jupyter_output><empty_output><jupyter_text>Quantitative evaluationPerturb each test sample and compare the clean and perturbed accuracies. $L_2$ perturbations:<jupyter_code>perturbed_accuracy_l2 = predict_model(model, mnist_testset, batch_size=test_batch_size, attack_function=fast_gradient_attack, attack_args=attack_args_l2)<jupyter_output><empty_output><jupyter_text>$L_\infty$ perturbations:<jupyter_code>perturbed_accuracy_linf = predict_model(model, mnist_testset, batch_size=test_batch_size, attack_function=fast_gradient_attack, attack_args=attack_args_linf)<jupyter_output><empty_output><jupyter_text>Your values for `clean_accuracy` and `perturbed_accuracy` should roughly match the ones below, even though they will of course not be identical.<jupyter_code>clean_accuracy
perturbed_accuracy_l2
perturbed_accuracy_linf<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1436232612703201, 'fraction_numerical': 0.011060834590246356, 'mean_word_length': 7.323570432357044, 'pattern_counts': {'":': 4, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '460225', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1970, 'n_tokens_neox': 1857, 'n_words': 484, 'path': 'part_1_training.ipynb', 'repo_name': 'spKay18/interactive_latencies', 'size': '5967'}