<jupyter_start><jupyter_text>Remove and Retrain  DatasetFor this experiment we have used Cifar10, while in the originl paper Cifar100 was used.Below there are a few images that we will use in this experiment.<jupyter_code># load dataset
train_loader, test_loader = load_cifar_dataset(data_path, config)
# display one batch
batch, target = next(iter(test_loader))

(ch, r, c) = batch[0].size()

fig=plt.figure(figsize=(10, 10))
columns = 3
rows = 3
for idx in range(1, columns*rows +1):
    image = unnormalize(batch[idx,:,:,:])
    img = [[[image[i, k, j] for i in range(ch)] for j in range (c)] for k in range(r)]
    ax = fig.add_subplot(rows, columns, idx)
    ax.title.set_text(labels[target[idx]])
    plt.imshow(img)

plt.show()<jupyter_output>/home/anca/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py:206: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  mean = torch.tensor(mean, dtype=torch.float32)
/home/anca/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py:207: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  std = torch.tensor(std, dtype=torch.float32)
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clippi[...]<jupyter_text>Initial training<jupyter_code># really short training session

# VGG11 neural network
initial_model = init_model(config)

train(initial_model, config, data_loader=train_loader, plot_name="temp_initial_model")

initial_accuracy_mean, initial_accuracy_std = test(initial_model, config, data_loader=test_loader)
print(initial_accuracy_mean, initial_accuracy_std)
# Load and display already trained model
initial_model = torch.load(os.path.join("results", "remove_and_retrain",
                                        "models", "initial_model"))
img = mpimg.imread(os.path.join("results", "remove_and_retrain",
                                "plots", "initial_model.jpg"))
fig=plt.figure(figsize=(20, 20))
imgplot = plt.imshow(img)
plt.axis('off')
plt.show()<jupyter_output><empty_output><jupyter_text>Create Modified dataset<jupyter_code># compute and save modified dataset
compute_modified_datasets(train_loader, test_loader, os.path.join(data_path, "temp"), config)
# Visualize modified dataset
#90% of the pixels are removed
k = "921" 
batch = torch.load(os.path.join("dataset", "modified_cifar_10", "FullGrad", k, "test", "batch_input0"))
target = torch.load(os.path.join("dataset", "modified_cifar_10", "FullGrad", k, "test", "batch_target0"))
(ch, r, c) = batch[0].size()

fig=plt.figure(figsize=(10, 10))
columns = 3
rows = 3
for idx in range(1, columns*rows +1):
    image = unnormalize(batch[idx,:,:,:])
    img = [[[image[i, k, j] for i in range(ch)] for j in range (c)] for k in range(r)]
    ax = fig.add_subplot(rows, columns, idx)
    ax.title.set_text(labels[target[idx]])
    plt.imshow(img)

plt.show()<jupyter_output>/home/anca/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py:206: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  mean = torch.tensor(mean, dtype=torch.float32)
/home/anca/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py:207: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  std = torch.tensor(std, dtype=torch.float32)
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clippi[...]<jupyter_text>Train on modified dataset<jupyter_code># Reads the previously modified datasets and retrains the models
remove_and_retrain(data_path, config, addition="temp_")
# Load the plots of an already trained models
initial_model = torch.load(os.path.join("results", "remove_and_retrain",
                                        "models", "initial_model"))
img = mpimg.imread(os.path.join("results", "remove_and_retrain",
                                "plots", "FullGrad_0.5.jpg"))
fig=plt.figure(figsize=(20, 20))
imgplot = plt.imshow(img)
plt.axis('off')
plt.show()<jupyter_output><empty_output><jupyter_text>Final result<jupyter_code># Load the final results
initial_model = torch.load(os.path.join("results", "remove_and_retrain",
                                        "models", "initial_model"))
img = mpimg.imread(os.path.join("results", "remove_and_retrain", "final_plot.jpeg"))
fig=plt.figure(figsize=(20, 20))
imgplot = plt.imshow(img)
plt.axis('off')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13460118168389956, 'fraction_numerical': 0.021048744460856722, 'mean_word_length': 4.837284482758621, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '54398', 'license': 'MIT', 'n_tokens_mistral': 1839, 'n_tokens_neox': 1744, 'n_words': 467, 'path': 'remove_and_retrain.ipynb', 'repo_name': 'rmleushuis/FACT', 'size': '5416'}