<jupyter_start><jupyter_text>Udacity - Machine Learning Engineer Nanodegree Capstone Project Title: Development of a LSTM Network to Predict Students’ Answers on Exam Questions  Implementation of DKT: Part 1: Define constants<jupyter_code>dataset = "data/ASSISTments_skill_builder_data.csv" # Dataset path
best_model_file = "saved_models/ASSISTments.best.model.weights.hdf5" # File to save the model.
train_log = "logs/dktmodel.train.log" # File to save the training log.
eval_log = "logs/dktmodel.eval.log" # File to save the testing log.
optimizer = "adagrad" # Optimizer to use
lstm_units = 250 # Number of LSTM units
batch_size = 20 # Batch size
epochs = 100 # Number of epochs to train
dropout_rate = 0.6 # Dropout rate
verbose = 1 # Verbose = {0,1,2}
testing_rate = 0.2 # Portion of data to be used for testing
validation_rate = 0.2 # Portion of training data to be used for validation<jupyter_output><empty_output><jupyter_text>Part 2: Pre-processing<jupyter_code>from Utils import *

dataset, num_skills = read_file(dataset)
X_train, X_val, X_test, y_train, y_val, y_test = split_dataset(dataset, validation_rate, testing_rate)

print("======== Data Summary ========")
print("Data size: %d" % len(dataset))
print("Training data size: %d" % len(X_train))
print("Validation data size: %d" % len(X_val))
print("Testing data size: %d" % len(X_test))
print("Number of skills: %d" % num_skills)
print("==============================")<jupyter_output>======== Data Summary ========
Data size: 4163
Training data size: 2665
Validation data size: 666
Testing data size: 832
Number of skills: 123
==============================<jupyter_text>Part 3: Building the model<jupyter_code>from StudentModel import DKTModel, DataGenerator

# Create generators for training/testing/validation
train_gen = DataGenerator(X_train[0:10], y_train[0:10], num_skills, batch_size)
val_gen = DataGenerator(X_val[0:10], y_val[0:10], num_skills, batch_size)
test_gen = DataGenerator(X_test[0:10], y_test[0:10], num_skills, batch_size)

# Create model
student_model = DKTModel(num_skills=train_gen.num_skills,
                      num_features=train_gen.feature_dim,
                      optimizer=optimizer,
                      hidden_units=lstm_units,
                      batch_size=batch_size,
                      dropout_rate=dropout_rate)<jupyter_output><empty_output><jupyter_text>Part 4: Train the Model<jupyter_code>history = student_model.fit(train_gen,
                  epochs=epochs,
                  val_gen=val_gen,
                  verbose=verbose,
                  filepath_bestmodel=best_model_file,
                  filepath_log=train_log)<jupyter_output>==== Training Started ====
Epoch 1/100

Epoch 00001: val_loss improved from inf to 0.65795, saving model to saved_models/ASSISTments.best.model.weights.hdf5
1/1 [==============================] - 9s 9s/step - loss: 0.6943 - val_loss: 0.6579 - val_auc: 0.5877 - val_acc: 0.6632 - val_pre: 0.7033
Epoch 2/100

Epoch 00002: val_loss did not improve
1/1 [==============================] - 6s 6s/step - loss: 0.6322 - val_loss: 0.7457 - val_auc: 0.5287 - val_acc: 0.5522 - val_pre: 0.6992
Epoch 3/100

Epoch 00003: val_loss improved from 0.65795 to 0.62957, saving model to saved_models/ASSISTments.best.model.weights.hdf5
1/1 [==============================] - 6s 6s/step - loss: 0.7717 - val_loss: 0.6296 - val_auc: 0.6043 - val_acc: 0.6577 - val_pre: 0.7002
Epoch 4/100

Epoch 00004: val_loss did not improve
1/1 [==============================] - 6s 6s/step - loss: 0.6662 - val_loss: 0.6965 - val_auc: 0.5630 - val_acc: 0.6309 - val_pre: 0.7076
Epoch 5/100

Epoch 00005: val_loss improved from 0.6295[...]<jupyter_text>Part 5: Load the Model with the Best Validation Loss<jupyter_code>student_model.load_weights(best_model_file)<jupyter_output><empty_output><jupyter_text>Part 6: Test the Model<jupyter_code>result = student_model.evaluate(test_gen, metrics=['auc','acc','pre'], verbose=verbose, filepath_log=eval_log)<jupyter_output>==== Evaluation Started ====
1/1 [==============================] - 1s 1s/step - pre: 0.7790 - acc: 0.7493 - auc: 0.5760
==== Evaluation Done ====
{'fraction_non_alphanumeric': 0.16155517990823473, 'fraction_numerical': 0.06399420429847863, 'mean_word_length': 4.493368700265252, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '15670', 'license': 'MIT', 'n_tokens_mistral': 1611, 'n_tokens_neox': 1402, 'n_words': 352, 'path': 'src/DKT.ipynb', 'repo_name': 'Wasim37/Deep-Knowledge-Tracing', 'size': '4141'}