<jupyter_start><jupyter_text>Save 예제> - 출처: http://paiai.tistory.com/26 [YOLO :)]<jupyter_code>import tensorflow as tf
input_data = [[ 1, 5, 3, 7, 8, 10, 12],
             [5, 8, 10, 3, 9, 7, 1]]
label_data = [[ 0, 0, 0, 1, 0], [1, 0, 0, 0, 0]]
INPUT_SIZE = 7
HIDDEN1_SIZE = 10
HIDDEN2_SIZE = 8
CLASSES = 5

epochs = 1000
Learning_Rate = 0.05
x = tf.placeholder(tf.float32, shape=[None, INPUT_SIZE], name='x')
y_ = tf.placeholder(tf.float32, shape=[None, CLASSES], name='y_')
tensor_map = {x: input_data, y_: label_data}
tensor_map<jupyter_output><empty_output><jupyter_text>모델과 weight 초기화 부분은 분리해서 코드를 짜는 것이 좋다.<jupyter_code>W_h1 = tf.Variable(tf.truncated_normal(shape=[INPUT_SIZE, HIDDEN1_SIZE]), dtype=tf.float32, name='W_h1')
b_h1 = tf.Variable(tf.zeros(shape=[HIDDEN1_SIZE]), dtype=tf.float32, name='b_h1')

W_h2 = tf.Variable(tf.truncated_normal(shape=[HIDDEN1_SIZE, HIDDEN2_SIZE], dtype=tf.float32, name='W_h2'))
b_h2 = tf.Variable(tf.zeros(shape=[HIDDEN2_SIZE]), dtype=tf.float32, name='b_h2')

W_o = tf.Variable(tf.truncated_normal(shape=[HIDDEN2_SIZE, CLASSES]), dtype=tf.float32, name='W_o')
b_o = tf.Variable(tf.zeros(shape=[CLASSES]), dtype=tf.float32, name='b_o')

param_list = [W_h1, b_h1, W_h2, b_h2, W_o, b_o]
# weight 가 끝나는 시점에서 saver 생성 - 그 이후에 넣으면 필요없는 variable까지 저장한다
saver = tf.train.Saver(param_list)
hidden1 = tf.sigmoid(tf.matmul(x, W_h1) + b_h1, name='hidden1')
hidden2 = tf.sigmoid(tf.matmul(hidden1, W_h2) + b_h2, name='hidden2')
y = tf.sigmoid(tf.matmul(hidden2, W_o) + b_o, name='y')
# cross entropy
cost = tf.reduce_sum(-y_*tf.log(y)-(1-y_)*tf.log(1-y), reduction_indices=1)
cost = tf.reduce_mean(cost)
train = tf.train.GradientDescentOptimizer(Learning_Rate).minimize(cost)
comp_pred = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(comp_pred, tf.float32))
sess = tf.Session()
sess.run(tf.global_variables_initializer())

# restore는 initialize 대신 하면 된다
# saver.restore(sess, './tf_model.ckpt')
# result = sess.run(y, tensor_map)

for i in range(epochs):
    _, loss, acc = sess.run([train, cost, accuracy], tensor_map)
    
    if i % 100 == 0:


        saver.save(sess, 'tf_model/tf_model.ckpt')


        print('----------------------')
        print("step       :", i)
        print("loss       : {0:.5f}".format(loss))
        print("accuracy   :", acc)<jupyter_output>----------------------
step       : 0
loss       : 5.49891
accuracy   : 0.0
----------------------
step       : 100
loss       : 0.83122
accuracy   : 1.0
----------------------
step       : 200
loss       : 0.45426
accuracy   : 1.0
----------------------
step       : 300
loss       : 0.29933
accuracy   : 1.0
----------------------
step       : 400
loss       : 0.21721
accuracy   : 1.0
----------------------
step       : 500
loss       : 0.16759
accuracy   : 1.0
----------------------
step       : 600
loss       : 0.13495
accuracy   : 1.0
----------------------
step       : 700
loss       : 0.11216
accuracy   : 1.0
----------------------
step       : 800
loss       : 0.09549
accuracy   : 1.0
----------------------
step       : 900
loss       : 0.08285
accuracy   : 1.0<jupyter_text>Restore 예제  training 코드 부분은 다시 training 하지 않는 경우 없어도 된다.<jupyter_code>cost = tf.reduce_sum(-y_*tf.log(y)-(1-y_)*tf.log(1-y), reduction_indices=1)
cost = tf.reduce_mean(cost)
train = tf.train.GradientDescentOptimizer(Learning_Rate).minimize(cost)
comp_pred = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(comp_pred, tf.float32))
sess = tf.Session()
#sess.run(tf.global_variables_initializer())
saver.restore(sess, 'tf_model/tf_model.ckpt')
result = sess.run(y, tensor_map)
print(result)<jupyter_output>INFO:tensorflow:Restoring parameters from tf_model/tf_model.ckpt
[[ 0.03673964  0.01228184  0.01713125  0.96912795  0.01401264]
 [ 0.97715771  0.00291012  0.00238059  0.02147566  0.00282537]]<jupyter_text>저장했던 weight 로 다시 트레이닝을 하면 결과 값이 loss가 낮고, accuracy가 높다.<jupyter_code>for i in range(epochs):

    _, loss, acc = sess.run([train, cost, accuracy], tensor_map)
    if i % 100 == 0:
        #saver.save(sess, './tf_model/tf_model.ckpt')
        print('----------------------')
        print("step       :", i)
        print("loss       : {0:.5f}".format(loss))
        print("accuracy   :", acc)<jupyter_output>----------------------
step       : 0
loss       : 0.08274
accuracy   : 1.0
----------------------
step       : 100
loss       : 0.07290
accuracy   : 1.0
----------------------
step       : 200
loss       : 0.06504
accuracy   : 1.0
----------------------
step       : 300
loss       : 0.05862
accuracy   : 1.0
----------------------
step       : 400
loss       : 0.05330
accuracy   : 1.0
----------------------
step       : 500
loss       : 0.04883
accuracy   : 1.0
----------------------
step       : 600
loss       : 0.04501
accuracy   : 1.0
----------------------
step       : 700
loss       : 0.04172
accuracy   : 1.0
----------------------
step       : 800
loss       : 0.03886
accuracy   : 1.0
----------------------
step       : 900
loss       : 0.03635
accuracy   : 1.0<jupyter_text>checkpoint 파일에 있는 weight 보는 방법<jupyter_code># checkpoint 파일에 저장된 weight 보는 방법
from tensorflow.python.tools import inspect_checkpoint as chkp
chkp.print_tensors_in_checkpoint_file('tf_model/tf_model.ckpt', tensor_name='', all_tensors=True)<jupyter_output>tensor_name:  Variable
[[ 0.38079959  0.83630204  1.93567228  0.66555643 -0.74033761  0.66069984
   0.12239929  0.37367296]
 [-1.00211251  1.42941451 -1.92600632 -1.07085252  0.13144255 -0.11721206
  -0.67877024  0.83177394]
 [-2.6683948   1.17708826 -1.24111366 -0.23206459 -1.57147813 -0.74702543
  -0.81409609 -0.0715283 ]
 [-0.21670227 -0.21233237  1.38918996  0.61057568 -1.90969586 -0.61162299
   0.39615303  0.45776087]
 [ 0.9065755  -0.35080519  1.47089553 -1.98241258 -0.21401995  0.14934178
   1.2331444  -0.40077686]
 [-0.00332462  0.02434572  0.53586984 -0.68623918 -1.46012354 -0.19449413
   0.01564419  0.37834615]
 [-1.65093923  0.09048604  0.42327902 -0.66795939  0.3204093  -0.45377421
   1.71553457  0.52760488]
 [ 1.16093564 -0.97796839  0.53956938 -1.11420357  1.13341153  1.79293895
   0.21121225 -0.21112512]
 [ 1.20019877  0.84878039  1.37371922  1.6986748  -0.04443777 -0.48825884
   0.25883052  1.06390357]
 [ 1.11508799  0.4229289  -0.35749397 -1.25006342 -1.16844678 -2.460[...]
{'fraction_non_alphanumeric': 0.20885375494071146, 'fraction_numerical': 0.17533596837944665, 'mean_word_length': 2.3137768465165007, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '33979', 'license': 'MIT', 'n_tokens_mistral': 3313, 'n_tokens_neox': 2712, 'n_words': 485, 'path': 'Python/basic/step10ModelExportImport.ipynb', 'repo_name': 'statKim/TIL', 'size': '6325'}