Mistral-7B-v0.1_colaMistral_scratch_cola

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4281
• Accuracy: {'accuracy': 0.8387850467289719}
• Matthews Correlation: 0.6114

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 2
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Matthews Correlation
1.9322	0.17	20	1.5215	{'accuracy': 0.5743048897411314}	0.0818
1.1953	0.33	40	0.9950	{'accuracy': 0.660594439117929}	0.1870
0.6611	0.5	60	0.7549	{'accuracy': 0.7353787152444871}	0.3527
0.6165	0.66	80	0.6317	{'accuracy': 0.7583892617449665}	0.4081
0.5467	0.83	100	0.5667	{'accuracy': 0.7842761265580057}	0.5041
0.4864	1.0	120	0.5268	{'accuracy': 0.7996164908916586}	0.5385
0.478	1.16	140	0.4803	{'accuracy': 0.8283796740172579}	0.5859
0.439	1.33	160	0.4965	{'accuracy': 0.8293384467881112}	0.5818
0.4395	1.49	180	0.4669	{'accuracy': 0.8283796740172579}	0.5778
0.4202	1.66	200	0.5002	{'accuracy': 0.825503355704698}	0.6192
0.3485	1.83	220	0.4360	{'accuracy': 0.8389261744966443}	0.6099
0.442	1.99	240	0.4391	{'accuracy': 0.840843720038351}	0.6121
0.3752	2.16	260	0.4306	{'accuracy': 0.8446788111217641}	0.6474
0.3013	2.32	280	0.4163	{'accuracy': 0.8427612655800575}	0.6216
0.3395	2.49	300	0.4151	{'accuracy': 0.8542665388302972}	0.6592
0.3305	2.66	320	0.4096	{'accuracy': 0.8475551294343241}	0.6299
0.342	2.82	340	0.4101	{'accuracy': 0.8465963566634708}	0.6322
0.3183	2.99	360	0.4166	{'accuracy': 0.8494726749760306}	0.6364
0.2551	3.15	380	0.4321	{'accuracy': 0.8542665388302972}	0.6503

Framework versions

• PEFT 0.7.1
• Transformers 4.36.2
• Pytorch 2.1.2+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0