beit-large-patch16-224-pt22k-finetuned-galaxy10-decals

This model is a fine-tuned version of microsoft/beit-large-patch16-224-pt22k on the matthieulel/galaxy10_decals dataset. It achieves the following results on the evaluation set:

• Loss: 0.5047
• Accuracy: 0.8771
• Precision: 0.8770
• Recall: 0.8771
• F1: 0.8764

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
1.5632	0.99	62	1.3358	0.5265	0.5377	0.5265	0.4840
0.8801	2.0	125	0.7053	0.7717	0.7710	0.7717	0.7559
0.7408	2.99	187	0.5995	0.7897	0.7878	0.7897	0.7803
0.6124	4.0	250	0.5448	0.8140	0.8178	0.8140	0.8076
0.5799	4.99	312	0.5354	0.8174	0.8224	0.8174	0.8165
0.567	6.0	375	0.5044	0.8247	0.8314	0.8247	0.8194
0.5237	6.99	437	0.4913	0.8388	0.8429	0.8388	0.8371
0.4674	8.0	500	0.4927	0.8484	0.8541	0.8484	0.8477
0.4869	8.99	562	0.4167	0.8546	0.8570	0.8546	0.8526
0.4442	10.0	625	0.4086	0.8579	0.8583	0.8579	0.8564
0.4294	10.99	687	0.4743	0.8489	0.8516	0.8489	0.8489
0.4032	12.0	750	0.4350	0.8664	0.8651	0.8664	0.8647
0.4028	12.99	812	0.4443	0.8568	0.8623	0.8568	0.8561
0.3939	14.0	875	0.4193	0.8608	0.8605	0.8608	0.8593
0.3447	14.99	937	0.4289	0.8698	0.8692	0.8698	0.8688
0.354	16.0	1000	0.4471	0.8653	0.8661	0.8653	0.8648
0.2934	16.99	1062	0.4888	0.8574	0.8573	0.8574	0.8546
0.3262	18.0	1125	0.4605	0.8602	0.8602	0.8602	0.8588
0.3287	18.99	1187	0.4439	0.8681	0.8682	0.8681	0.8673
0.2848	20.0	1250	0.4986	0.8641	0.8633	0.8641	0.8615
0.283	20.99	1312	0.4663	0.8692	0.8681	0.8692	0.8676
0.3106	22.0	1375	0.4668	0.8720	0.8735	0.8720	0.8697
0.2785	22.99	1437	0.4899	0.8664	0.8649	0.8664	0.8650
0.2635	24.0	1500	0.5047	0.8771	0.8770	0.8771	0.8764
0.2573	24.99	1562	0.5144	0.8732	0.8730	0.8732	0.8723
0.238	26.0	1625	0.5012	0.8732	0.8729	0.8732	0.8723
0.2358	26.99	1687	0.5021	0.8681	0.8709	0.8681	0.8690
0.2624	28.0	1750	0.5154	0.8715	0.8711	0.8715	0.8705
0.229	28.99	1812	0.5087	0.8698	0.8690	0.8698	0.8689
0.227	29.76	1860	0.5104	0.8726	0.8725	0.8726	0.8718

Framework versions

• Transformers 4.37.2
• Pytorch 2.3.0
• Datasets 2.19.1
• Tokenizers 0.15.1