<jupyter_start><jupyter_text>This notebook trains a default MNIST model for comparisonThis notebook trains the same MNIST model used in `with_prior.ipynb`, with the same hyper-parameters. These hyper-parameters were chosen to perform well on the model in this notebook. In other words, they were optimized for the model without attribution priors.<jupyter_code>import numpy as np
import pandas as pd
import altair as alt
import tensorflow as tf
import shap

FLAGS = tf.app.flags.FLAGS
tf.app.flags.DEFINE_string('f', None, 'Define to make the jupyter kernel happy.')
import mnist_train
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
FLAGS.lamb = 0.0
FLAGS.epochs = 50
save_dir = 'without_prior/'<jupyter_output><empty_output><jupyter_text>Input PipelineLike before, we use `mnist_model.py` and `mnist_train.py` to define and train the graph<jupyter_code>cond_input_op, x, y, y_, train_pl, loss, train_step, eg_loss, eg_train, train_eg, accuracy_op, accuracy_update_op, reset_metrics_op, \
    training_iterator, validation_iterator, test_iterator, handle = mnist_train.pipeline()
training_handle   = sess.run(training_iterator.string_handle())
validation_handle = sess.run(validation_iterator.string_handle())
test_handle       = sess.run(test_iterator.string_handle())
validation_total_variances, validation_accuracies, test_accuracy = mnist_train.train(save_dir, sess, cond_input_op, y, train_pl, loss, \
                                                                                     train_step, eg_loss, eg_train, train_eg, accuracy_op, \
                                                                                     accuracy_update_op, reset_metrics_op, \
                                                                                     training_iterator, validation_iterator, test_iterator, handle)<jupyter_output>INFO:tensorflow:Restoring parameters from without_prior/tion vald (batch): 768.316711
Test accuracy: 0.995200<jupyter_text>ResultsBelow, we plot the training curves and the expected gradients pixel attributions for a sample of 5 test images. We can see that the attributions, while meaningful, are somewhat noisy.<jupyter_code>data = pd.DataFrame({
    'Iteration': mnist_train.validation_step * np.arange(len(validation_total_variances[1:])),
    'Total Variance of Attributions': validation_total_variances[1:],
    'Validation Accuracy': validation_accuracies[1:]
})
base = alt.Chart(data).mark_line().encode(
    alt.X('Iteration:Q')
)
base.encode(alt.Y('Validation Accuracy:Q', scale=alt.Scale(zero=False))) | base.encode(alt.Y('Total Variance of Attributions:Q'))
train_data = []
sess.run(training_iterator.initializer)
for i in range(10):
    train_data.append(sess.run(cond_input_op, feed_dict={handle: training_handle}))
train_data = np.concatenate(train_data, axis=0)

sess.run(test_iterator.initializer)
test_data = sess.run(cond_input_op, feed_dict={handle: test_handle})<jupyter_output><empty_output><jupyter_text>Attribution MapsThe plot below shows attribution maps for five test images, for each possible class. Red pixels contribute posivitely to the indicated image being part of the listed class, while blue pixels contribute negatively to the indicated class. For example, the top left image indicates which pixels make the first image (7) look like a 0.<jupyter_code>labels = np.array([['Label: {}'.format(i) for i in range(10)]] + [[''] * 10] * 4)
explainer = shap.GradientExplainer((cond_input_op, y), data=train_data, session=sess)
shap_values = explainer.shap_values(test_data[:5])
shap.image_plot(shap_values, -test_data[:5], labels=labels)<jupyter_output>Using TensorFlow backend.<jupyter_text>Evaluation on NoiseHere we evaluate the model on noisy test data to simulate a domain shift. That is, we randomly select roughly a fraction of pixels and flip them to be white or black with equal probability. We then re-evaluate our model on the noisy test set. The baseline model breaks down quickly with pixel noise.<jupyter_code>def salt_and_pepper(image_pl, minval=0.0, maxval=1.0, percent_flipped=0.1):
    z = 1.0 - (percent_flipped) * 0.5
    t  = tf.distributions.Normal(0.0, 1.0).quantile(z)
    
    random_normal  = tf.random_normal(shape=tf.shape(image_pl))
    clipped_normal = tf.cast(tf.greater(random_normal, t), tf.float32) - \
                     tf.cast(tf.less(random_normal, -t), tf.float32)
    max_range = maxval - minval
    clipped_normal = clipped_normal * max_range
    
    aug_im = image_pl + clipped_normal
    aug_im = tf.clip_by_value(aug_im, 
                              clip_value_min=minval,
                              clip_value_max=maxval)
    return aug_im
noise_level = tf.placeholder(tf.float32, ())
noisy_x = salt_and_pepper(x, minval=-0.5, maxval=0.5, percent_flipped=noise_level)
def get_noise_test_perf(percent_flipped):
    sess.run(reset_metrics_op)
    sess.run(test_iterator.initializer)
    while True:
        try:
            image_batch, label_batch = sess.run([noisy_x, y_], feed_dict={handle: test_handle,
                                                                          noise_level: percent_flipped})
            sess.run(accuracy_update_op, feed_dict={handle: test_handle,
                                                    x: image_batch,
                                                    y_: label_batch})
        except tf.errors.OutOfRangeError:
            break
    test_accuracy = sess.run(accuracy_op)
    return test_accuracy
flip_values = [0.0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3]
noise_perf  = []
for flip_val in flip_values:
    print('Running noise level {}...'.format(flip_val), end='\r')
    noise_perf.append(get_noise_test_perf(flip_val))
data = pd.DataFrame({'Percent of Pixels Flipped': flip_values, 'Test Accuracy': noise_perf})
alt.Chart(data).mark_line().encode(
    alt.X('Percent of Pixels Flipped:Q'),
    alt.Y('Test Accuracy:Q')
).properties(title='Test Performance of the Baseline Model on Noisy Test Data')
import matplotlib.pyplot as plt

fig, axs = plt.subplots(1, 7, figsize=(21, 3))

for i in range(7):
    flip_val = flip_values[i]
    sess.run(test_iterator.initializer)
    image_batch = sess.run(noisy_x, feed_dict={handle: test_handle, noise_level: flip_val})
    axs[i].axis('off')
    axs[i].set_title('{}% noise'.format(int(flip_val * 100)))
    axs[i].imshow(-image_batch[0, :, :, 0], cmap='gray')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10127171215880894, 'fraction_numerical': 0.013647642679900745, 'mean_word_length': 3.557597173144876, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '134688', 'license': 'MIT', 'n_tokens_mistral': 2074, 'n_tokens_neox': 1983, 'n_words': 498, 'path': 'examples/mnist/without_prior.ipynb', 'repo_name': 'ch6845/attributionpriors', 'size': '6448'}