beit-base-patch16-224-ve-U13-b-80

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.6397
• Accuracy: 0.8478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.3187	0.4565
1.6193	2.0	13	1.3087	0.4565
1.6193	2.92	19	1.2939	0.4565
1.6044	4.0	26	1.2802	0.4565
1.5061	4.92	32	1.2867	0.4565
1.5061	6.0	39	1.2813	0.4565
1.3879	6.92	45	1.2511	0.4565
1.3007	8.0	52	1.1294	0.5652
1.3007	8.92	58	1.0096	0.5435
1.1213	10.0	65	0.9308	0.5217
0.9968	10.92	71	0.9280	0.5435
0.9968	12.0	78	0.8034	0.6087
0.8771	12.92	84	0.7791	0.6522
0.7383	14.0	91	0.8005	0.6739
0.7383	14.92	97	0.7408	0.7391
0.6658	16.0	104	0.9305	0.6304
0.5879	16.92	110	0.7136	0.7609
0.5879	18.0	117	0.7106	0.7609
0.4609	18.92	123	0.6998	0.6957
0.4123	20.0	130	0.7931	0.7609
0.4123	20.92	136	0.9417	0.6739
0.3552	22.0	143	0.7868	0.7174
0.3552	22.92	149	0.9073	0.6957
0.2896	24.0	156	0.8542	0.7174
0.2316	24.92	162	0.7159	0.7391
0.2316	26.0	169	0.7219	0.7174
0.2339	26.92	175	0.7071	0.7609
0.2055	28.0	182	1.0110	0.6739
0.2055	28.92	188	0.6397	0.8478
0.1995	30.0	195	0.6922	0.8478
0.169	30.92	201	0.6171	0.8478
0.169	32.0	208	0.6632	0.8261
0.1586	32.92	214	0.6475	0.8261
0.1439	34.0	221	0.8332	0.6957
0.1439	34.92	227	0.6816	0.7826
0.1698	36.0	234	0.8066	0.7609
0.1362	36.92	240	0.7150	0.8043
0.1362	38.0	247	0.7193	0.8043
0.1344	38.92	253	0.8181	0.7609
0.1317	40.0	260	0.6547	0.8261
0.1317	40.92	266	0.8459	0.7609
0.123	42.0	273	0.7700	0.8261
0.123	42.92	279	0.9338	0.7391
0.102	44.0	286	0.8536	0.8043
0.1015	44.92	292	0.9725	0.7391
0.1015	46.0	299	0.8865	0.8043
0.1313	46.92	305	0.8947	0.8261
0.1312	48.0	312	0.8235	0.8043
0.1312	48.92	318	0.7326	0.8261
0.1168	50.0	325	0.8654	0.7609
0.09	50.92	331	0.7645	0.8261
0.09	52.0	338	0.7632	0.8478
0.0872	52.92	344	0.7496	0.8043
0.0813	54.0	351	0.8846	0.8043
0.0813	54.92	357	0.9214	0.7826
0.0955	56.0	364	0.9284	0.7826
0.1031	56.92	370	0.8855	0.7826
0.1031	58.0	377	0.8619	0.8043
0.0962	58.92	383	0.8187	0.8261
0.0891	60.0	390	0.7430	0.8478
0.0891	60.92	396	0.7530	0.8478
0.0679	62.0	403	0.7790	0.8261
0.0679	62.92	409	0.7905	0.8261
0.0805	64.0	416	0.8286	0.8261
0.0619	64.92	422	0.8371	0.8043
0.0619	66.0	429	0.8655	0.8043
0.0778	66.92	435	0.8897	0.8043
0.0712	68.0	442	0.9385	0.8043
0.0712	68.92	448	0.9611	0.8043
0.0659	70.0	455	0.9597	0.8043
0.0602	70.92	461	0.9635	0.8043
0.0602	72.0	468	0.9733	0.8043
0.0641	72.92	474	0.9754	0.8043
0.0653	73.85	480	0.9753	0.8043

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0