<jupyter_start><jupyter_text>Gutenberg Chapters Dataset with Hierarchical Attention Network (HAN)HAN analysis with the h2 hierarchical Gutenberg chapters dataset. Using the following configuration:1. Using Learned Embedding1. Term Embedding size: 1001. Sentence Embedding size: 2001. Document Embedding size: 3001. Top vocabulary count 5,0001. Adam Learning Rate of 1e-41. Gradient Constraint: 5.0<jupyter_code>%matplotlib inline

import os
import sys
ai_lit_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir, os.pardir, os.pardir))
print("Loading AI Lit system from path", ai_lit_path)
sys.path.append(ai_lit_path)

import numpy as np
import pandas as pd
import tensorflow as tf

from ai_lit.analysis import analysis_util
from ai_lit.input.gutenberg_dataset import gb_input
from ai_lit.university.gutenberg import gb_chap_han
dataset_wkspc = os.path.join(ai_lit_path, 'workspace', 'gb_input')
training_wkspc = os.path.join(ai_lit_path, 'workspace', 'gutenberg_chapters')
subjects = gb_input.get_subjects(dataset_wkspc)
# use the flags imported from the univserity and the model to set the configuration
tf.flags.FLAGS.epochs=10
tf.flags.FLAGS.batch_queue_capacity=200
model_name = 'han_100-200-300'
evaluation_name = 'standard_eval'
univ = gb_chap_han.GbChaptersHANUniversity(model_name, training_wkspc, dataset_wkspc)
accuracy, f1, cm = analysis_util.train_and_evaluate(univ, model_name, evaluation_name)
# use the flags imported from the univserity and the model to set the configuration
tf.flags.FLAGS.epochs=10
tf.flags.FLAGS.batch_queue_capacity=100
tf.flags.FLAGS.max_grad_norm=0.5

model_name = 'han_100-200-300_v2'
evaluation_name = 'standard_eval'
univ = gb_chap_han.GbChaptersHANUniversity(model_name, training_wkspc, dataset_wkspc)
accuracy, f1, cm = analysis_util.train_and_evaluate(univ, model_name, evaluation_name)
# use the flags imported from the univserity and the model to set the configuration
tf.flags.FLAGS.epochs=10
tf.flags.FLAGS.term_embedding_size=50
tf.flags.FLAGS.sentence_embedding_size=50
tf.flags.FLAGS.document_embedding_size=50
tf.flags.FLAGS.batch_queue_capacity=1024
tf.flags.FLAGS.max_grad_norm=1
tf.flags.FLAGS.batch_size=36

model_name = 'han_100-200-300_v3'
evaluation_name = 'standard_eval'
univ = gb_chap_han.GbChaptersHANUniversity(model_name, training_wkspc, dataset_wkspc)
accuracy, f1, cm = analysis_util.train_and_evaluate(univ, model_name, evaluation_name)
# use the flags imported from the univserity and the model to set the configuration
tf.flags.FLAGS.epochs=10
tf.flags.FLAGS.term_embedding_size=50
tf.flags.FLAGS.sentence_embedding_size=100
tf.flags.FLAGS.document_embedding_size=150
tf.flags.FLAGS.batch_queue_capacity=1024
tf.flags.FLAGS.max_grad_norm=5
tf.flags.FLAGS.batch_size=36
tf.flags.FLAGS.learning_rate=1e-3

model_name = 'han_100-200-300_v4'
evaluation_name = 'standard_eval'
univ = gb_chap_han.GbChaptersHANUniversity(model_name, training_wkspc, dataset_wkspc)
accuracy, f1, cm = analysis_util.train_and_evaluate(univ, model_name, evaluation_name)
evaluation_name = 'book_aggregation_eval'
univ = gb_chap_han.GbChaptersHANUniversity(model_name, training_wkspc, dataset_wkspc)
accuracy, f1, cm = analysis_util.train_and_evaluate(univ, model_name, evaluation_name, **{"aggr_books": True})<jupyter_output>Validation Step 544
Targets:
[1 2 2 0 2 3 1 2 3 5 4 3 3 3 2 1 0 3 2 2 1 4 0 4 1 3 1 1 1 3 4 2 1 3 5 4]
Predictions:
[1 3 1 0 3 1 1 1 3 1 4 4 3 4 1 4 0 1 1 1 4 4 0 2 3 3 1 1 1 1 4 1 1 3 3 3]
Loss 1.98298, Acc 0.444444

Testing examples exhausted
Shutting down all network threads
Checking validation mode... {'aggr_books': True}
Evaluating the full book aggregation.
Accuracy: 0.674928503337
F1: 0.600151340021
{'fraction_non_alphanumeric': 0.08299430431244914, 'fraction_numerical': 0.062381339842690536, 'mean_word_length': 6.390781563126253, 'pattern_counts': {'":': 1, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '262527', 'license': 'MIT', 'n_tokens_mistral': 1529, 'n_tokens_neox': 1329, 'n_words': 347, 'path': 'ai_lit/analysis/gutenberg/gb_chapters_han_learned_embeddings_1.ipynb', 'repo_name': 'LincLabUCCS/Genre_Identification', 'size': '3687'}