mistral-journal-finetune

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7152

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss
1.4632	1.0870	25	0.8646
0.6844	2.1739	50	0.8295
0.3071	3.2609	75	0.8981
0.2151	4.3478	100	0.9571
0.1535	5.4348	125	1.0427
0.1188	6.5217	150	1.2999
0.1234	7.6087	175	1.1580
0.1023	8.6957	200	1.4546
0.0956	9.7826	225	1.4925
0.0915	10.8696	250	1.5539
0.0892	11.9565	275	1.5340
0.0873	13.0435	300	1.5638
0.0805	14.1304	325	1.5932
0.0816	15.2174	350	1.6178
0.0791	16.3043	375	1.6719
0.0799	17.3913	400	1.6869
0.0775	18.4783	425	1.6920
0.08	19.5652	450	1.6984
0.0737	20.6522	475	1.7132
0.0797	21.7391	500	1.7152

Framework versions

• PEFT 0.11.2.dev0
• Transformers 4.42.0.dev0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1