swin-tiny-patch4-window7-224-dmae-va-U5-42B

This model is a fine-tuned version of microsoft/swin-tiny-patch4-window7-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7189
• Accuracy: 0.7667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.6573	0.1167
1.5376	1.94	15	1.5974	0.1167
1.4718	2.97	23	1.4168	0.1167
1.3557	4.0	31	1.2830	0.45
1.3557	4.9	38	1.2193	0.45
1.2189	5.94	46	1.0932	0.5
1.1132	6.97	54	0.9622	0.65
0.9671	8.0	62	0.8943	0.6167
0.9671	8.9	69	0.7767	0.7
0.8402	9.94	77	0.8288	0.6167
0.7153	10.97	85	0.7101	0.7167
0.64	12.0	93	0.7146	0.75
0.5653	12.9	100	0.8075	0.6667
0.5653	13.94	108	0.6274	0.75
0.4914	14.97	116	0.6952	0.7167
0.4542	16.0	124	0.9034	0.6333
0.4263	16.9	131	0.7039	0.7333
0.4263	17.94	139	0.7514	0.7
0.3796	18.97	147	0.8074	0.7
0.3455	20.0	155	0.7189	0.7667
0.2995	20.9	162	0.7582	0.7333
0.3143	21.94	170	0.7938	0.7167
0.3143	22.97	178	0.8181	0.7
0.2768	24.0	186	0.8045	0.7333
0.3009	24.9	193	0.7281	0.7167
0.2419	25.94	201	0.8014	0.7
0.2419	26.97	209	0.8608	0.7333
0.2528	28.0	217	0.9163	0.7
0.2257	28.9	224	0.8970	0.7167
0.2133	29.94	232	0.9016	0.7333
0.2113	30.97	240	0.9065	0.7167
0.2113	32.0	248	0.9403	0.75
0.2005	32.9	255	0.9440	0.7333
0.1825	33.94	263	0.9553	0.7167
0.2109	34.97	271	0.9528	0.7
0.2109	36.0	279	0.9380	0.7
0.1755	36.9	286	0.9309	0.7
0.1859	37.94	294	0.9335	0.7

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0