<jupyter_start><jupyter_text>Logistic Regression with MNIST* [참고: TensorFlow.org](https://www.tensorflow.org/get_started/mnist/beginners)* [소스: mnist_softmax.py in verion 1.4](https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/examples/tutorials/mnist/mnist_softmax.py)  Import modules<jupyter_code>"""A very simple MNIST classifier.
See extensive documentation at
https://www.tensorflow.org/get_started/mnist/beginners in version 1.4
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import time
import numpy as np

import tensorflow as tf

sess_config = tf.ConfigProto(gpu_options=tf.GPUOptions(allow_growth=True))

np.random.seed(219)
tf.set_random_seed(219)<jupyter_output><empty_output><jupyter_text>Import data<jupyter_code># Load training and eval data from tf.keras
(train_data, train_labels), (test_data, test_labels) = \
    tf.keras.datasets.mnist.load_data()

train_data = train_data / 255.
train_data = train_data.reshape(-1, 784)
train_labels = np.asarray(train_labels, dtype=np.int32)

test_data = test_data / 255.
test_data = test_data.reshape(-1, 784)
test_labels = np.asarray(test_labels, dtype=np.int32)<jupyter_output><empty_output><jupyter_text>Show the MNIST<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

index = 1000
print("label = {}".format(train_labels[index]))
plt.imshow(train_data[index].reshape(28, 28), cmap='gray')
plt.show()<jupyter_output><empty_output><jupyter_text>Build a model Create placeholders for inputs and labels<jupyter_code># 여기를 직접 채워 넣으시면 됩니다.
# x, y는 배치 데이터를 받을 수 있는 placeholder
# x: inputs
x = tf.placeholder(tf.float32, name='x', shape=[None, 784])
# y: labels
y = tf.placeholder(tf.int32, name='y', shape=[None])<jupyter_output><empty_output><jupyter_text>Create weight and bias<jupyter_code># 여기를 직접 채워 넣으시면 됩니다.
# create Variables using `get_variable`
W = tf.get_variable(name='weight', shape=[784, 10],
                    initializer=tf.random_normal_initializer())
b = tf.get_variable(name='bias', shape=[10],
                    initializer=tf.zeros_initializer())<jupyter_output><empty_output><jupyter_text>Build a model: $y = Wx + b$<jupyter_code># 여기를 직접 채워 넣으시면 됩니다.
y_pred = tf.matmul(x, W) + b<jupyter_output><empty_output><jupyter_text>Define loss function* [`tf.nn.softmax_cross_entropy_with_logits_v2`](https://www.tensorflow.org/api_docs/python/tf/nn/softmax_cross_entropy_with_logits_v2)* [`tf.losses.softmax_cross_entropy`](https://www.tensorflow.org/api_docs/python/tf/losses/softmax_cross_entropy)<jupyter_code># 출처: TensorFlow 공식 문서
# The raw formulation of cross-entropy,
#
#   tf.reduce_mean(-tf.reduce_sum(y * tf.log(tf.nn.softmax(y_pred)),
#                                 reduction_indices=[1]))
#
# can be numerically unstable.
#
# So here we use tf.nn.softmax_cross_entropy_with_logits on the raw
# outputs of 'y', and then average across the batch.
y_one_hot = tf.one_hot(y, depth=10)
cross_entropy = tf.losses.softmax_cross_entropy(onehot_labels=y_one_hot,
                                                logits=y_pred)<jupyter_output><empty_output><jupyter_text>Create a optimizer<jupyter_code>train_op = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)<jupyter_output><empty_output><jupyter_text>`tf.Session()` and train<jupyter_code>batch_size = 32

sess = tf.Session(config=sess_config)
sess.run(tf.global_variables_initializer())

num_batches_per_epoch = int(len(train_data) / batch_size)
print("num_batches_per_epoch {}".format(num_batches_per_epoch))

max_epochs = 1 # Train for only ten epochs
losses = []
start_time = time.time()
for epoch in range(max_epochs):
  # 여기를 직접 채워 넣으시면 됩니다.
  batch_index = np.random.permutation(len(train_data))
  for step in range(num_batches_per_epoch):
    # 여기를 직접 채워 넣으시면 됩니다.
    # train_data, train_labels의 인덱스를 잘 설정해보세요.
    batch_xs = train_data[step*batch_size:(step+1)*batch_size]
    batch_ys = train_labels[step*batch_size:(step+1)*batch_size]
    _, loss = sess.run([train_op, cross_entropy],
                       feed_dict={x: batch_xs, y: batch_ys})
    losses.append(loss)
  
    if step % 100 == 0:
      print("step: {}, loss: {}".format(step, loss))
    
print('training done!')
print("Elapsed time: {}".format(time.time() - start_time))<jupyter_output><empty_output><jupyter_text>Plot the loss funtion<jupyter_code>plt.plot(losses, label='loss')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Test trained model* test accuracy: 0.8893<jupyter_code>correct_prediction = tf.equal(tf.argmax(y_pred, 1), tf.cast(y, tf.int64))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(sess.run(accuracy, feed_dict={x: test_data,
                                    y: test_labels}))
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

test_batch_size = 16
batch_index = np.random.choice(len(test_data), size=test_batch_size, replace=False)
batch_xs = test_data[batch_index]
y_pred_ = sess.run(y_pred, feed_dict={x: batch_xs})

fig = plt.figure(figsize=(16, 10))
for i, (px, py) in enumerate(zip(batch_xs, y_pred_)):
  p = fig.add_subplot(4, 8, i+1)
  p.set_title("y_pred: {}".format(np.argmax(py)))
  p.imshow(px.reshape(28, 28), cmap='gray')
  p.axis('off')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13055764883195178, 'fraction_numerical': 0.01714393368500377, 'mean_word_length': 5.396385542168675, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 5, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '581669', 'license': 'Apache-2.0', 'n_tokens_mistral': 2127, 'n_tokens_neox': 2110, 'n_words': 353, 'path': 'week02/06_logistic_regression.ipynb', 'repo_name': 'modulabs/modu-tensorflow', 'size': '5308'}