<jupyter_start><jupyter_text>Loading the model.<jupyter_code># Restore model.
sess = tf.Session()    
#First let's load meta graph and restore weights
saver = tf.train.import_meta_graph('tf_mnist_model_layers/tf_mnist_model.ckpt.meta')
saver.restore(sess,tf.train.latest_checkpoint('tf_mnist_model_layers/'))<jupyter_output>INFO:tensorflow:Restoring parameters from tf_mnist_model_layers/tf_mnist_model.ckpt<jupyter_text>Loading the dataset<jupyter_code>dataset = mnist.load_data()
train_data = dataset[0][0] / 255
train_data = train_data[..., np.newaxis].astype('float32')
train_labels = np_utils.to_categorical(dataset[0][1]).astype('float32')
test_data = dataset[1][0] / 255
test_data = test_data[..., np.newaxis].astype('float32')
test_labels = np_utils.to_categorical(dataset[1][1]).astype('float32')
plt.imshow(train_data[0, ..., 0])
# Get the first input tensor.
network_input = sess.graph.get_operations()[0].outputs[0]
network_input<jupyter_output><empty_output><jupyter_text>The TensorFlow graph Look at all operations to find out the ones that are interesting for weights/activations.<jupyter_code>sess.graph.get_operations()<jupyter_output><empty_output><jupyter_text>TensorFlow exposes a graph API in which nodes correspond to operations and directed edges correspond to to tensors flowing from one operation to the next. Operations know their respective input and output tensors. Tensors know by which operations they get produced and for which operations they serve as input. c.f. https://www.tensorflow.org/extend/tool_developers/<jupyter_code>conv_op = sess.graph.get_operation_by_name('conv2d_1/convolution')
print('inputs: ', list(conv_op.inputs))
print('outputs: ', list(conv_op.outputs))
weight_tensor = conv_op.inputs[1]
weight_tensor
# Producer ops for the tensor.
weight_tensor.op
# Consumer ops for the tensor.
weight_tensor.consumers()<jupyter_output><empty_output><jupyter_text>Getting the weightsFor getting the weights we need to run the output tensor of operations that read the weights or the seconde input tensor of the actual MatMul/Conv operations.<jupyter_code>weight_read_op = sess.graph.get_operation_by_name('conv2d_1/kernel/read')
weight_read_op

conv_op = sess.graph.get_operation_by_name('conv2d_1/convolution')
conv_op
weights_1 = sess.run(weight_read_op.outputs[0])
weights_2 = sess.run(conv_op.inputs[-1])
np.all(weights_1 == weights_2)<jupyter_output><empty_output><jupyter_text>Looking at the actual kernels.<jupyter_code>print(weights_1.shape)
for i in range(32):
    plt.imshow(weights_1[..., 0, i])
    plt.show()
weights_1[..., 0, 0]<jupyter_output><empty_output><jupyter_text>The same goes for the bias.<jupyter_code>bias_add_op = sess.graph.get_operation_by_name('conv2d_1/BiasAdd')
bias_add_op
bias = sess.run(bias_add_op.inputs[-1])
bias<jupyter_output><empty_output><jupyter_text>Getting the activations and net inputsActivations and net inputs can be found as the output tensors of the respective operations.Slices of the example code here are used to unit test the `network` package.<jupyter_code>activation_op = sess.graph.get_operation_by_name('conv2d_1/Relu')
activation_op.outputs[0]
activations = sess.run(activation_op.outputs[0], feed_dict={network_input: test_data[0:1]})
activations.shape<jupyter_output><empty_output><jupyter_text>Look at the activations<jupyter_code>for i in range(32):
    plt.imshow(activations[0, ..., i])
    plt.show()
activations[0, 5:10, 5:10, 0]
net_input_op = sess.graph.get_operation_by_name('conv2d_1/BiasAdd')
net_input_op.outputs[0]
net_inputs = sess.run(net_input_op.outputs[0], feed_dict={network_input: test_data[0:1]})
net_inputs.shape
for i in range(32):
    plt.imshow(net_inputs[0, ..., i])
    plt.show()
net_inputs[0, 5:10, 5:10, 0]<jupyter_output><empty_output><jupyter_text>Getting input and output shapeFor each tensor the shape can be retrieved as a `TensorShape` object or as a list.<jupyter_code>conv_op.inputs[0].shape
conv_op.inputs[0].shape.as_list()<jupyter_output><empty_output><jupyter_text>Getting layer propertiesInternally TensorFlow represents graph as protobufs that only define nodes with their inputs. This definition can either be retrieved as globally `graph.graph_def` as as `op.node_def` for every single operation.<jupyter_code>sess.graph_def
activation_op.node_def<jupyter_output><empty_output><jupyter_text>Kernel sizeKernel size is not defined in the protobuf, but has to be read from the weight tensor shape.<jupyter_code>read_op = sess.graph.get_operation_by_name('conv2d_2/kernel/read')
read_op.outputs[0].shape.as_list()[:2]<jupyter_output><empty_output><jupyter_text>Padding<jupyter_code>conv_op.node_def
# The padding value has to be decoded from bytes into a string.
conv_op.node_def.attr['padding'].s.decode('utf8')<jupyter_output><empty_output><jupyter_text>Strides<jupyter_code>strides = conv_op.node_def.attr['strides']
strides.list.i[1], strides.list.i[2]<jupyter_output><empty_output><jupyter_text>Pool sizePool size is defined as `ksize` in the maxpool definition.<jupyter_code>max_pool_op = sess.graph.get_operation_by_name('max_pooling2d_1/MaxPool')
max_pool_op.node_def
kernel = max_pool_op.node_def.attr['ksize']
kernel.list.i[1], kernel.list.i[2]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11321476381717345, 'fraction_numerical': 0.01855039204436795, 'mean_word_length': 7.616144975288303, 'pattern_counts': {'":': 0, '<': 60, '<?xml version=': 0, '>': 60, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '642664', 'license': 'MIT', 'n_tokens_mistral': 1877, 'n_tokens_neox': 1809, 'n_words': 374, 'path': 'notebooks/tensorflow/introspection.ipynb', 'repo_name': 'Petr-By/qtpyvis', 'size': '5229'}