<jupyter_start><jupyter_text>Getting started with TensorFlow Load data Load the MNIST data using helper functions in tensorFlow. It will download the data if not already present in your computer.<jupyter_code>from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)<jupyter_output>Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz<jupyter_text>The dataset contain images of digits and corresponding labels. They are already separated into training, testing and validation sets.<jupyter_code>print "Image dimension = ", mnist.train.images.shape, "Labels dimension = ", mnist.train.labels.shape<jupyter_output>Image dimension =  (55000, 784) Labels dimension =  (55000, 10)<jupyter_text>Train a basic softmax RegressionThe MNIST images have 28x28 = 784 dimension, but the dataset has been flattened for purposes of training a softmax regression. This will destroy the spatial structure present in the image. We will exploit the spatial structure when we use convolutional nets. This is a classic case where a softmax regression is a natural, simple model. If you want to assign probabilities to an object being one of several different things, softmax is the thing to do. Even later on, when we train more sophisticated models, the final step will be a layer of softmax.Very simply, we compute the evidence that any image $x$ belongs to class $i$ as \begin{equation*}Evidence_i = \sum_j{W_{i,j}x_j+b_i} \end{equation*}where $W_{i,j}$ is the weight and $b_i$ are the biases which we will learn during the training phase. Next, we use the softmax as an activation or link function to convert our linear evidence to probability distribution over 10 classes. Softmax is a simple function which is \begin{equation*}softmax(x)_i = \frac{\exp{x_i}}{\sum_j{\exp{x_j}}}\end{equation*}   Built the model We build a symbolic model using inbuilt functions in tensorflow. For defining input to the model, we can use placeholder variables with specified dimensions.<jupyter_code>import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])
y_ = tf.placeholder(tf.float32, [None, 10])
W = tf.Variable(tf.zeros([784,10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x,W)+b)<jupyter_output><empty_output><jupyter_text>Cost FunctionTo train a model, we need to define a **Cost or Loss Function** which we minimize during training to learn the weights and biases. The cost function we will use here is the _Cross Entropy_ function. \begin{equation*}H_{\tilde{y}}(y) = -\sum_i{\tilde{y}_i\log{y_i}}\end{equation*}where $y$ is our predicted class probabilities, and $\tilde{y}$ is the true class probability (the one-hot vector we'll input).<jupyter_code>cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)<jupyter_output><empty_output><jupyter_text>Running the model in a session<jupyter_code>sess = tf.Session()
init = tf.initialize_all_variables()
sess.run(init)

num_steps = 2000
for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})<jupyter_output><empty_output><jupyter_text>Evaluating the model<jupyter_code>correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))<jupyter_output>0.9177
{'fraction_non_alphanumeric': 0.09937038050917055, 'fraction_numerical': 0.019436079934300576, 'mean_word_length': 6.108949416342412, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '350121', 'license': 'Apache-2.0', 'n_tokens_mistral': 1220, 'n_tokens_neox': 1144, 'n_words': 373, 'path': 'softmax-regression.ipynb', 'repo_name': 'mayankgrd/tensorflow-tutorial', 'size': '3653'}