<jupyter_start><jupyter_text>MNIST data set with softmax regression  Data Retrieve the data set.  This data set consists of scanned images of digits, and as label the digit they represent. The task is to train a neural network to recognize handwritten digits.<jupyter_code>from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data/', one_hot=True)<jupyter_output>Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz<jupyter_text>Import the modules required.<jupyter_code>import tensorflow as tf<jupyter_output><empty_output><jupyter_text>Model The input data are $28 \times 28$ pixel images, where each pixel is an intensity between 0 and 1.  The output is a label, one for each digit, i.e., $0$ to $9$. The network is defined as $y = softmax(x \cdot W + b)$, where $x$ represents the pixels of the image ($28 \times 28 = 784$), $y$ represents the probabilities of the 10 digits, $W$ represents the weights of the network ($784 \times 10$ and $b$ is the bias for each digit ($10$). Define a placeholder for the input data.<jupyter_code>x = tf.placeholder(tf.float32, [None, 28*28])<jupyter_output><empty_output><jupyter_text>Define the variables to be determined by the training algorithm, the weights $W$, and the bias $b$.<jupyter_code>W = tf.Variable(tf.zeros([28*28, 10]))
b = tf.Variable(tf.zeros([10]))<jupyter_output><empty_output><jupyter_text>Define the network model.<jupyter_code>y = tf.nn.softmax(tf.matmul(x, W) + b)<jupyter_output><empty_output><jupyter_text>Loss function and training We define the loss function as the cross entropy $H_{y'}(y) = -\sum_i y'_i \log y_i$. Define a placeholder for the labels.<jupyter_code>y_ = tf.placeholder(tf.float32, [None, 10])<jupyter_output><empty_output><jupyter_text>Define the cross entropy.<jupyter_code>cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))<jupyter_output><empty_output><jupyter_text>A training step consists of a backpropagation step that minimizes the loss function.<jupyter_code>train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)<jupyter_output><empty_output><jupyter_text>Define the initialization, initializing all the variables.<jupyter_code>init = tf.global_variables_initializer()<jupyter_output><empty_output><jupyter_text>Define a session to run the computations.<jupyter_code>session = tf.Session()<jupyter_output><empty_output><jupyter_text>Perform the initialization of the variables.<jupyter_code>session.run(init)<jupyter_output><empty_output><jupyter_text>Train the model by using 1000 batches of 100 training examples each.<jupyter_code>for i in range(1000):
  batch_xs, batch_ys = mnist.train.next_batch(100)
  session.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})<jupyter_output><empty_output><jupyter_text>Evaluation What is the accuracy of our model? A correct prediction means that the model assigns the highest probability to the digit that is the label in our test set.<jupyter_code>correct_predictions = tf.equal(tf.arg_max(y, 1), tf.arg_max(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_predictions, tf.float32))<jupyter_output><empty_output><jupyter_text>Run the model on the test data to get the accuracy as defined above.<jupyter_code>print(session.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))<jupyter_output>0.9169
{'fraction_non_alphanumeric': 0.10587899543378995, 'fraction_numerical': 0.021689497716894976, 'mean_word_length': 7.132250580046404, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 55, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '179904', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1251, 'n_tokens_neox': 1168, 'n_words': 314, 'path': 'Python/TensorFlow/mnist_tutorial.ipynb', 'repo_name': 'Gjacquenot/training-material', 'size': '3504'}