vit-base-patch16-224-U8-40b

This model is a fine-tuned version of google/vit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5666
• Accuracy: 0.8824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.3457	1.0	20	1.3070	0.4706
1.1498	2.0	40	1.0956	0.5686
0.8293	3.0	60	0.8270	0.6471
0.5448	4.0	80	0.6145	0.8235
0.3525	5.0	100	0.6439	0.7451
0.2436	6.0	120	0.5427	0.8235
0.195	7.0	140	0.6276	0.7843
0.1629	8.0	160	0.7868	0.7255
0.1697	9.0	180	0.8245	0.7255
0.1324	10.0	200	0.6599	0.8235
0.1714	11.0	220	0.7453	0.7647
0.0908	12.0	240	0.5666	0.8824
0.0812	13.0	260	0.9997	0.7451
0.0672	14.0	280	0.8049	0.8039
0.0843	15.0	300	0.6723	0.8431
0.0946	16.0	320	0.8892	0.7451
0.0684	17.0	340	1.1429	0.7451
0.0711	18.0	360	1.1384	0.7451
0.0677	19.0	380	1.0296	0.7843
0.0562	20.0	400	0.9803	0.7647
0.0688	21.0	420	0.9401	0.7843
0.0576	22.0	440	1.0823	0.7843
0.0892	23.0	460	1.0819	0.7255
0.063	24.0	480	1.0756	0.7647
0.055	25.0	500	0.9693	0.7647
0.0407	26.0	520	1.0132	0.7451
0.0562	27.0	540	1.0267	0.7843
0.0365	28.0	560	1.0530	0.7451
0.0363	29.0	580	0.9277	0.7843
0.0392	30.0	600	0.9798	0.8039
0.0374	31.0	620	1.0239	0.8039
0.0386	32.0	640	1.0221	0.8039
0.0345	33.0	660	1.0239	0.7843
0.035	34.0	680	1.0163	0.8039
0.0367	35.0	700	1.0902	0.8039
0.0219	36.0	720	1.1079	0.7843
0.0263	37.0	740	1.0727	0.8039
0.0261	38.0	760	1.0471	0.8039
0.0193	39.0	780	1.0347	0.8039
0.0301	40.0	800	1.0319	0.8039

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0