<jupyter_start><jupyter_text>Interactive Tensorflow MNIST demo  In this tutorial, we will show you how to use Interactive Tensorflow to change the hyper-parameters of a Neural Network while it is training on the MNIST dataset. You can use it as a reference for the integration of Interactive Tensorflow into your own project. ![learning_rate_plot](images/learning_rate_plot.png) **Import** Tensorflow and the Interactive Tensorflow server.<jupyter_code>import os
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

import tfset.server<jupyter_output><empty_output><jupyter_text>Create **placeholders** for the Tensors we want to change.<jupyter_code># create tensors for parameters
learning_rate = tf.get_variable("learning_rate", initializer=tf.constant(0.1, dtype=tf.float32))
dropout_prob = tf.get_variable("dropout_prob", initializer=tf.constant(0.9, dtype=tf.float32))
is_training = tf.placeholder(tf.bool, [])<jupyter_output><empty_output><jupyter_text>[MNIST](http://yann.lecun.com/exdb/mnist/) is a dataset of handwritten digits that is often used as the "Hello World" of Machine Learning. The task is to classify which digit (1 to 9) appears in each image. See examples of four digits below. ![abc](images/MNIST.png) Set some parameters and **import MNIST**.<jupyter_code># set some parameters
num_iterations = 10000000
mnist_data_path = "demo/data/mnist"
summary_path = "demo/summary"

# import MNIST
mnist = input_data.read_data_sets(mnist_data_path, one_hot=True)<jupyter_output><empty_output><jupyter_text>Define a graph of a simple **Convolutional Neural Network**. The ConvNet is regularized with [Dropout](https://www.tensorflow.org/api_docs/python/tf/layers/dropout) which we will control together with the learning rate using Interactive Tensorflow.<jupyter_code># create the model (a simple convolutional network)
x = tf.placeholder(tf.float32, (None, 784))
y = tf.placeholder(tf.float32, (None, 10))

x_reshaped = tf.reshape(x, (-1, 28, 28, 1))

conv1 = tf.layers.conv2d(x_reshaped, filters=32, kernel_size=(3, 3), strides=(1, 1), activation=tf.nn.relu)
pool1 = tf.layers.max_pooling2d(conv1, pool_size=(2, 2), strides=(2, 2))

conv2 = tf.layers.conv2d(pool1, filters=64, kernel_size=(3, 3), strides=(1, 1), activation=tf.nn.relu)
pool2 = tf.layers.max_pooling2d(conv2, pool_size=(2, 2), strides=(2, 2))

conv3 = tf.layers.conv2d(pool2, filters=128, kernel_size=(3, 3), strides=(1, 1), activation=tf.nn.relu)
pool4 = tf.reduce_mean(conv3, reduction_indices=(1, 2))

dropout = tf.layers.dropout(pool4, rate=dropout_prob, training=is_training)

logits = tf.layers.dense(dropout, 10)

cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=logits))
train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))<jupyter_output><empty_output><jupyter_text>Create summary ops for training and test accuracy as well as our hyper-parameters. We can visualize these summaries using [TensorBoard](https://www.tensorflow.org/get_started/summaries_and_tensorboard).<jupyter_code># create summary ops
train_summaries = tf.summary.merge([tf.summary.scalar("train_accuracy", accuracy),
                                    tf.summary.scalar("train_loss", cross_entropy),
                                    tf.summary.scalar("learning_rate", learning_rate)])
test_summaries = tf.summary.merge([tf.summary.scalar("test_accuracy", accuracy)])

# create a summary writer
summary_writer = tf.summary.FileWriter(summary_path)<jupyter_output><empty_output><jupyter_text>Run the **training session**.<jupyter_code># clean old tensorboard summaries
files = os.listdir(summary_path)
for file in files:
  os.remove(os.path.join(summary_path, file))


with tf.Session() as sess:
  sess.run(tf.global_variables_initializer())

  # start the server
  s, thread = server.run_server([learning_rate, dropout_prob], sess)

  # training
  print("Training started.")
    
  for step in range(num_iterations):
    batch_xs, batch_ys = mnist.train.next_batch(64)

    _, train_summary = sess.run([train_step, train_summaries], feed_dict={
      x: batch_xs,
      y: batch_ys,
      is_training: True
    })
    summary_writer.add_summary(train_summary, global_step=step)

    if step % 100 == 0 and step > 0:
      # check events
      s.check_events(step)

      # test model
      test_summary = sess.run(test_summaries, feed_dict={
        x: mnist.test.images,
        y: mnist.test.labels,
        is_training: False
      })
      summary_writer.add_summary(test_summary, global_step=step)

  # evaluation
  accuracy = sess.run(accuracy, feed_dict={
    x: mnist.test.images,
    y: mnist.test.labels,
    is_training: False
  })

print("\nTrained for %d iterations" % num_iterations)
print("Accuracy: %.2f%%" % (accuracy * 100))

s.shutdown()
thread.join(timeout=10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11789347695990425, 'fraction_numerical': 0.01954917215240375, 'mean_word_length': 5.220843672456576, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 26, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '592410', 'license': 'MIT', 'n_tokens_mistral': 1759, 'n_tokens_neox': 1654, 'n_words': 388, 'path': 'tfset/MNIST_demo.ipynb', 'repo_name': 'ondrejba/interactive-tensorflow', 'size': '5013'}