led-risalah_data_v4

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6965
• Rouge1 Precision: 0.7537
• Rouge1 Recall: 0.2044
• Rouge1 Fmeasure: 0.3201

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1 Fmeasure	Rouge1 Precision	Rouge1 Recall
2.4649	0.91	8	1.9501	0.2231	0.5607	0.1407
1.7599	1.94	17	1.7553	0.2741	0.657	0.1746
1.4655	2.97	26	1.6912	0.2786	0.6685	0.1774
1.2734	4.0	35	1.7006	0.2589	0.651	0.162
1.2852	4.91	43	1.6481	0.2733	0.6657	0.1732
1.1964	5.94	52	1.6380	0.263	0.6567	0.1655
1.108	6.97	61	1.6441	0.2766	0.6757	0.1746
1.1023	8.91	72	1.1080	0.2842	0.6932	0.1794
1.2354	9.94	81	1.1105	0.2816	0.6858	0.1779
1.1152	10.97	90	1.1317	0.2872	0.71	0.1804
1.17	12.0	99	1.1206	0.2896	0.6942	0.1837
1.0691	12.91	107	1.1037	0.2941	0.7234	0.1851
0.9594	13.94	116	1.1145	0.2983	0.7299	0.1879
1.0332	14.97	125	1.1295	0.2959	0.7243	0.1863
0.9519	16.0	134	1.1271	0.2916	0.7114	0.1839
0.8779	16.91	142	1.1314	0.2971	0.7192	0.1878
0.944	18.91	152	0.8427	0.3212	0.7799	0.2036
0.9652	19.94	161	0.8398	0.3075	0.7396	0.1951
0.9622	20.97	170	0.8421	0.3255	0.7776	0.207
0.9645	22.0	179	0.8550	0.3045	0.7283	0.1934
0.8923	22.91	187	0.8556	0.3145	0.7585	0.1992
0.8635	23.94	196	0.8622	0.3086	0.7445	0.1957
0.827	24.97	205	0.8648	0.3047	0.7358	0.193
0.8529	26.0	214	0.8650	0.3129	0.7586	0.1981
0.7505	26.91	222	0.8719	0.3135	0.7591	0.1985
0.7491	27.94	231	0.8710	0.3078	0.7419	0.1951

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.15.1