w266_model2_BERT_LSTM_1

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.6673
• Accuracy: {'accuracy': 0.586}
• F1: {'f1': 0.5941271393567649}
• Precision: {'precision': 0.6305594263991693}
• Recall: {'recall': 0.586}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
No log	1.0	125	2.7886	{'accuracy': 0.563}	{'f1': 0.5737642190234387}	{'precision': 0.6070380044002861}	{'recall': 0.563}
No log	2.0	250	3.2762	{'accuracy': 0.567}	{'f1': 0.5732065475023022}	{'precision': 0.6124992011023714}	{'recall': 0.567}
No log	3.0	375	3.1370	{'accuracy': 0.57}	{'f1': 0.5799666523302439}	{'precision': 0.6122839339063632}	{'recall': 0.57}
0.0465	4.0	500	3.3590	{'accuracy': 0.569}	{'f1': 0.5796357806282344}	{'precision': 0.6093440842818532}	{'recall': 0.5689999999999998}
0.0465	5.0	625	3.4285	{'accuracy': 0.57}	{'f1': 0.580483223593091}	{'precision': 0.618976915416096}	{'recall': 0.57}

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.2
• Tokenizers 0.13.3