FPB_finetuned_v1

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4649
• Accuracy: 0.9303
• F1: 0.9303
• Precision: 0.9303
• Recall: 0.9303

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.7905	1.0	97	0.6913	0.7504	0.7471	0.7458	0.7504
0.3516	2.0	194	0.3914	0.8476	0.8480	0.8517	0.8476
0.2545	3.0	291	0.3302	0.8882	0.8870	0.8911	0.8882
0.1225	4.0	388	0.3488	0.8723	0.8730	0.8801	0.8723
0.0674	5.0	485	0.3910	0.8970	0.8961	0.8963	0.8970
0.0458	6.0	582	0.4545	0.9028	0.9022	0.9036	0.9028
0.0963	7.0	679	0.3467	0.9100	0.9100	0.9104	0.9100
0.0781	8.0	776	0.4528	0.8999	0.8991	0.8996	0.8999
0.0961	9.0	873	0.3966	0.9042	0.9049	0.9091	0.9042
0.0643	10.0	970	0.3486	0.9158	0.9159	0.9160	0.9158
0.0521	11.0	1067	0.5745	0.8955	0.8931	0.9030	0.8955
0.0162	12.0	1164	0.4968	0.9042	0.9047	0.9070	0.9042
0.0106	13.0	1261	0.4925	0.9158	0.9161	0.9171	0.9158
0.0056	14.0	1358	0.5128	0.9129	0.9126	0.9149	0.9129
0.0116	15.0	1455	0.4791	0.9202	0.9199	0.9197	0.9202
0.0004	16.0	1552	0.4417	0.9216	0.9214	0.9218	0.9216
0.0121	17.0	1649	0.4378	0.9202	0.9199	0.9205	0.9202
0.0003	18.0	1746	0.4624	0.9245	0.9245	0.9247	0.9245
0.0001	19.0	1843	0.4697	0.9274	0.9275	0.9277	0.9274
0.0001	20.0	1940	0.4649	0.9303	0.9303	0.9303	0.9303

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Tokenizers 0.15.2