<jupyter_start><jupyter_text>Training CNN. Skip next 2 cells if you want to test saved model. Training CNN has taken about 30 min with nVidia 940mx GPU.<jupyter_code>best_val_score = -9999999
early_stop_patience = 5
last_score = 0

%%time

with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for iteration in range(X_train.shape[0] // batch_size):
            X_batch = X_train[iteration * batch_size: iteration * batch_size + batch_size]
            y_batch = y_train[iteration * batch_size: iteration * batch_size + batch_size]
            sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        acc_train = accuracy.eval(feed_dict={X: X_batch, y: y_batch})
        acc_valid = accuracy.eval(feed_dict={X: X_valid, y: y_valid})
        print("Epoch:", epoch+1, "Train accuracy:", acc_train, "Val accuracy:", acc_valid)
        # save only best model
        if acc_valid > best_val_score:
            best_val_score = acc_valid
            save_path = saver.save(sess, "./saved_models/model1.3_tf")
            print("Model saved.")
        # early stopping
        if acc_valid < last_score:
            early_stop_patience -= 1
        if early_stop_patience <= 0:
            break<jupyter_output>Epoch: 1 Train accuracy: 0.453125 Val accuracy: 0.3628049
Model saved.
Epoch: 2 Train accuracy: 0.6875 Val accuracy: 0.6097561
Model saved.
Epoch: 3 Train accuracy: 0.78125 Val accuracy: 0.6707317
Model saved.
Epoch: 4 Train accuracy: 0.765625 Val accuracy: 0.7012195
Model saved.
Epoch: 5 Train accuracy: 0.796875 Val accuracy: 0.69512194
Epoch: 6 Train accuracy: 0.828125 Val accuracy: 0.72865856
Model saved.
Epoch: 7 Train accuracy: 0.765625 Val accuracy: 0.7134146
Epoch: 8 Train accuracy: 0.890625 Val accuracy: 0.722561
Epoch: 9 Train accuracy: 0.890625 Val accuracy: 0.7530488
Model saved.
Epoch: 10 Train accuracy: 0.875 Val accuracy: 0.76829267
Model saved.
Epoch: 11 Train accuracy: 0.984375 Val accuracy: 0.72865856
Epoch: 12 Train accuracy: 0.953125 Val accuracy: 0.75
Epoch: 13 Train accuracy: 0.96875 Val accuracy: 0.7652439
Epoch: 14 Train accuracy: 0.9375 Val accuracy: 0.7743902
Model saved.
Epoch: 15 Train accuracy: 0.96875 Val accuracy: 0.7835366
Model saved.
Wall time: 16min 58[...]<jupyter_text>Evaluate saved model<jupyter_code>def evaluate(X_test, y_test):
    with tf.Session() as sess:
        init = tf.global_variables_initializer()
        sess.run(init) 
        saver = tf.train.import_meta_graph('saved_models/model1.3_tf.meta')
        saver.restore(sess, tf.train.latest_checkpoint('./saved_models/'))
        pred = sess.run([accuracy, loss], feed_dict={X: X_test, y: y_test})
        print("Test accuracy =", pred[0])
        print("Test loss =", pred[1])

evaluate(X_test, y_test)<jupyter_output>INFO:tensorflow:Restoring parameters from ./saved_models/model1.3_tf
Test accuracy = 0.75147927
Test loss = 1.1124166
{'fraction_non_alphanumeric': 0.1039364640883978, 'fraction_numerical': 0.09737569060773481, 'mean_word_length': 3.562204724409449, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '154716', 'license': 'MIT', 'n_tokens_mistral': 1170, 'n_tokens_neox': 978, 'n_words': 284, 'path': 'model1.2_tensorflow.ipynb', 'repo_name': 'Vitalinsh/Roman-numbers-recognizer', 'size': '2896'}