bart-base-sci

This model is a fine-tuned version of facebook/bart-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.7325

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
3.5083	1.0	1452	3.0211
3.1105	2.0	2904	2.8653
2.9224	3.0	4356	2.7973
2.7795	4.0	5808	2.7556
2.7012	5.0	7260	2.7293
2.603	6.0	8712	2.7064
2.5325	7.0	10164	2.6993
2.4599	8.0	11616	2.6817
2.3871	9.0	13068	2.6872
2.3239	10.0	14520	2.6818
2.2717	11.0	15972	2.6759
2.2161	12.0	17424	2.6810
2.1472	13.0	18876	2.6958
2.1074	14.0	20328	2.6932
2.0596	15.0	21780	2.7109
2.0151	16.0	23232	2.7186
1.9662	17.0	24684	2.7245
1.9305	18.0	26136	2.7425
1.8754	19.0	27588	2.7616
1.8375	20.0	29040	2.7709
1.7806	21.0	30492	2.7838
1.7572	22.0	31944	2.7989
1.706	23.0	33396	2.8198
1.6636	24.0	34848	2.8418
1.6349	25.0	36300	2.8498
1.6061	26.0	37752	2.8751
1.5659	27.0	39204	2.8888
1.5266	28.0	40656	2.9037
1.4952	29.0	42108	2.9209
1.4684	30.0	43560	2.9456
1.4353	31.0	45012	2.9749
1.3855	32.0	46464	2.9816
1.3589	33.0	47916	3.0008
1.3192	34.0	49368	3.0116
1.2974	35.0	50820	3.0332
1.2678	36.0	52272	3.0592
1.2423	37.0	53724	3.0762
1.211	38.0	55176	3.0833
1.1914	39.0	56628	3.1130
1.1569	40.0	58080	3.1248
1.1258	41.0	59532	3.1385
1.0997	42.0	60984	3.1589
1.0776	43.0	62436	3.1805
1.0519	44.0	63888	3.1950
1.0278	45.0	65340	3.2122
1.0112	46.0	66792	3.2242
0.9883	47.0	68244	3.2453
0.9707	48.0	69696	3.2584
0.9486	49.0	71148	3.2716
0.9273	50.0	72600	3.2943
0.9074	51.0	74052	3.3075
0.8863	52.0	75504	3.3217
0.8615	53.0	76956	3.3410
0.8452	54.0	78408	3.3509
0.8268	55.0	79860	3.3673
0.8191	56.0	81312	3.3802
0.7964	57.0	82764	3.3924
0.7811	58.0	84216	3.4044
0.7629	59.0	85668	3.4228
0.7489	60.0	87120	3.4360
0.7358	61.0	88572	3.4527
0.7281	62.0	90024	3.4634
0.7043	63.0	91476	3.4811
0.6938	64.0	92928	3.4885
0.6802	65.0	94380	3.5052
0.6673	66.0	95832	3.5136
0.6553	67.0	97284	3.5237
0.6494	68.0	98736	3.5355
0.6388	69.0	100188	3.5463
0.629	70.0	101640	3.5605
0.6132	71.0	103092	3.5717
0.6018	72.0	104544	3.5811
0.5942	73.0	105996	3.5909
0.5834	74.0	107448	3.6005
0.5785	75.0	108900	3.6125
0.5669	76.0	110352	3.6177
0.5611	77.0	111804	3.6319
0.5532	78.0	113256	3.6357
0.5467	79.0	114708	3.6449
0.544	80.0	116160	3.6468
0.5334	81.0	117612	3.6577
0.5313	82.0	119064	3.6643
0.5241	83.0	120516	3.6741
0.5125	84.0	121968	3.6754
0.5118	85.0	123420	3.6858
0.5063	86.0	124872	3.6933
0.4992	87.0	126324	3.6969
0.4956	88.0	127776	3.7041
0.491	89.0	129228	3.7100
0.4862	90.0	130680	3.7109
0.4836	91.0	132132	3.7166
0.4801	92.0	133584	3.7171
0.4774	93.0	135036	3.7246
0.4738	94.0	136488	3.7251
0.4766	95.0	137940	3.7272
0.4694	96.0	139392	3.7312
0.4708	97.0	140844	3.7305
0.4673	98.0	142296	3.7320
0.4615	99.0	143748	3.7323
0.4622	100.0	145200	3.7325

Framework versions

• Transformers 4.43.4
• Pytorch 1.13.1
• Datasets 2.12.0
• Tokenizers 0.19.1