<jupyter_start><jupyter_text>使用RNN神经网络分类MNIST数据集  1.RNN网络示意图  1.1.RNN神经网络示意图 ![RNN神经网络结构图](./RNN.jpg "神经网络结构图")  1.2.LSTM神经元结构图 ![LSTM神经元结构图](./LSTM.jpg "LSTM神经元结构图")  1.3.RNN网络按照时间展开 ![RNN网络按照时间展开](./extend_rnn.jpg "RNN网络按照时间展开")  2.下载MNIST数据集<jupyter_code># 下载mnist数据集
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('/tmp/', one_hot=True)<jupyter_output>Extracting /tmp/train-images-idx3-ubyte.gz
Extracting /tmp/train-labels-idx1-ubyte.gz
Extracting /tmp/t10k-images-idx3-ubyte.gz
Extracting /tmp/t10k-labels-idx1-ubyte.gz<jupyter_text>3.定义RNN神经网络分类MNIST数据集  3.1.引入依赖包<jupyter_code>import numpy as np
import tensorflow as tf
sess = tf.InteractiveSession()<jupyter_output><empty_output><jupyter_text>3.2.定义模型参数<jupyter_code># Parameters
learning_rate = 0.001
training_iters = 100000
batch_size = 128
display_step = 10

# Network Parameters
n_input = 28 # MNIST data input (img shape: 28*28)
n_steps = 28 # timesteps
n_hidden = 128 # hidden layer num of features
n_classes = 10 # MNIST total classes (0-9 digits)

# tf Graph input
X = tf.placeholder("float", [None, n_steps, n_input])
Y = tf.placeholder("float", [None, n_classes])<jupyter_output><empty_output><jupyter_text>3.3.定义RNN神经网络模型<jupyter_code>def RecNeuralNetwork(x):
    # we need split our X into sequences
    data = tf.transpose(x, [1,0,2])
    data = tf.reshape(data, [-1, n_input])
    datas = tf.split(data, n_steps, axis=0)

    # 1.RNN Layer
    lstm_cell = tf.contrib.rnn.BasicLSTMCell(n_hidden)
    outputs, states = tf.contrib.rnn.static_rnn(lstm_cell, datas, dtype=tf.float32)
    
    ### basicly same as above (can see from the source code of static_rnn)###
    # from tensorflow.python.ops import variable_scope as vs
    # with tf.variable_scope("rnn") as varscope:
    #     if varscope.caching_device is None:
    #         varscope.set_caching_device(lambda op: op.device)
    #     lstm_cell = tf.contrib.rnn.BasicLSTMCell(n_hidden)
    #     state = lstm_cell.zero_state(batch_size, tf.float32) # the initial state of the lstm memeory state
    #     outputs = [];states = []
    #     for time, data in enumerate(datas):
    #         if time > 0: varscope.reuse_variables()
    #         (output, state) = lstm_cell(data, state)
    #         outputs.append(output)
    #        states.append(state)
    ### end ###
    
    # 2.Readout Layer
    W = tf.Variable(tf.random_normal([n_hidden, n_classes]))
    b = tf.Variable(tf.random_normal([n_classes]))
    output = tf.matmul(outputs[-1], W) + b # 这里并没有做softmax，那么在cost函数中，则使用tf.nn.softmax_cross_entropy_with_logits
    
    return output<jupyter_output><empty_output><jupyter_text>3.4.定义神经网络训练和评估方法<jupyter_code># train graph
predict    = RecNeuralNetwork(X)
cost       = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=predict, labels=Y))
train_step = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

# eval graph
correct  = tf.equal(tf.argmax(predict, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))<jupyter_output><empty_output><jupyter_text>3.5.训练测试<jupyter_code>init = tf.global_variables_initializer()
sess.run(init)
step = 1
# Keep training until reach max iterations
while step * batch_size < training_iters:
    batch_x, batch_y = mnist.train.next_batch(batch_size)
    # Reshape data to get 28 seq of 28 elements
    batch_x = batch_x.reshape((batch_size, n_steps, n_input))
    # Run optimization op (backprop)
    sess.run(train_step, feed_dict={X: batch_x, Y: batch_y})
    if step % display_step == 0:
        # Calculate batch accuracy
        acc = sess.run(accuracy, feed_dict={X: batch_x, Y: batch_y})
        # Calculate batch loss
        loss = sess.run(cost, feed_dict={X: batch_x, Y: batch_y})
        print "Iter " + str(step*batch_size) + ", Minibatch Loss= " + \
                  "{:.6f}".format(loss) + ", Training Accuracy= " + \
                  "{:.5f}".format(acc)
    step += 1
print "Optimization Finished!"

# Calculate accuracy for 128 mnist test images
test_len = 128
test_data = mnist.test.images[:test_len].reshape((-1, n_steps, n_input))
test_label = mnist.test.labels[:test_len]
print "Testing Accuracy:", sess.run(accuracy, feed_dict={X: test_data, Y: test_label})<jupyter_output>Iter 1280, Minibatch Loss= 1.871495, Training Accuracy= 0.35938
Iter 2560, Minibatch Loss= 1.546391, Training Accuracy= 0.52344
Iter 3840, Minibatch Loss= 1.225400, Training Accuracy= 0.54688
Iter 5120, Minibatch Loss= 1.023603, Training Accuracy= 0.62500
Iter 6400, Minibatch Loss= 0.822722, Training Accuracy= 0.71875
Iter 7680, Minibatch Loss= 1.103695, Training Accuracy= 0.59375
Iter 8960, Minibatch Loss= 0.868721, Training Accuracy= 0.72656
Iter 10240, Minibatch Loss= 0.601901, Training Accuracy= 0.82031
Iter 11520, Minibatch Loss= 0.398659, Training Accuracy= 0.87500
Iter 12800, Minibatch Loss= 0.716563, Training Accuracy= 0.75000
Iter 14080, Minibatch Loss= 0.485264, Training Accuracy= 0.84375
Iter 15360, Minibatch Loss= 0.366924, Training Accuracy= 0.88281
Iter 16640, Minibatch Loss= 0.470318, Training Accuracy= 0.90625
Iter 17920, Minibatch Loss= 0.265154, Training Accuracy= 0.89844
Iter 19200, Minibatch Loss= 0.264342, Training Accuracy= 0.90625
Iter 20480, Minibatch Loss= 0.15[...]
{'fraction_non_alphanumeric': 0.12113744075829384, 'fraction_numerical': 0.06843601895734597, 'mean_word_length': 4.340080971659919, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '114088', 'license': 'MIT', 'n_tokens_mistral': 2331, 'n_tokens_neox': 2109, 'n_words': 428, 'path': '03_rnn_mnist_classifier/rnn_mnist_classifier.ipynb', 'repo_name': 'bnusss/tensorflow_learning', 'size': '5275'}