vit-base-patch16-224-dmae-va-da-40

This model is a fine-tuned version of google/vit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2193
• Accuracy: 0.9302

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	3	1.3743	0.3256
No log	1.85	6	1.2281	0.3721
No log	2.77	9	1.0964	0.5116
1.2661	4.0	13	1.0032	0.5814
1.2661	4.92	16	0.8501	0.6744
1.2661	5.85	19	0.7082	0.7209
0.85	6.77	22	0.6280	0.7674
0.85	8.0	26	0.5146	0.8140
0.85	8.92	29	0.4501	0.8140
0.577	9.85	32	0.4339	0.8372
0.577	10.77	35	0.3841	0.8372
0.577	12.0	39	0.3444	0.8372
0.3806	12.92	42	0.3030	0.8605
0.3806	13.85	45	0.3079	0.8140
0.3806	14.77	48	0.3414	0.8140
0.2651	16.0	52	0.2920	0.8837
0.2651	16.92	55	0.3190	0.9070
0.2651	17.85	58	0.2844	0.8605
0.1992	18.77	61	0.2465	0.8837
0.1992	20.0	65	0.2368	0.8605
0.1992	20.92	68	0.2805	0.8605
0.1368	21.85	71	0.2958	0.8605
0.1368	22.77	74	0.3295	0.8837
0.1368	24.0	78	0.2933	0.8837
0.121	24.92	81	0.2677	0.9070
0.121	25.85	84	0.2436	0.8837
0.121	26.77	87	0.2420	0.8837
0.1046	28.0	91	0.2097	0.9070
0.1046	28.92	94	0.2193	0.9302
0.1046	29.85	97	0.2095	0.9302
0.1008	30.77	100	0.2074	0.9070
0.1008	32.0	104	0.2136	0.9070
0.1008	32.92	107	0.2177	0.9302
0.0796	33.85	110	0.2228	0.9302
0.0796	34.77	113	0.2289	0.9302
0.0796	36.0	117	0.2305	0.9302
0.0763	36.92	120	0.2298	0.9302

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1