NXAIR_M_mistral-7B

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6929

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00025
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
1.1435	0.0702	100	1.1971
1.0993	0.1404	200	1.0390
1.0643	0.2107	300	0.9309
0.956	0.2809	400	0.9125
0.9906	0.3511	500	0.8591
0.9083	0.4213	600	0.8703
0.8951	0.4916	700	0.8179
0.8352	0.5618	800	0.7852
0.8472	0.6320	900	0.7772
0.8733	0.7022	1000	0.7447
0.7958	0.7725	1100	0.7082
0.8726	0.8427	1200	0.7125
0.804	0.9129	1300	0.6909
0.8467	0.9831	1400	0.7287
0.4705	1.0534	1500	0.6921
0.4864	1.1236	1600	0.6648
0.4535	1.1938	1700	0.6765
0.4542	1.2640	1800	0.6620
0.4789	1.3343	1900	0.6584
0.5154	1.4045	2000	0.6492
0.459	1.4747	2100	0.6647
0.5168	1.5449	2200	0.6484
0.483	1.6152	2300	0.6795
0.4768	1.6854	2400	0.6730
0.4821	1.7556	2500	0.6404
0.4929	1.8258	2600	0.6409
0.5438	1.8961	2700	0.6551
0.4598	1.9663	2800	0.6740
0.4902	2.0365	2900	0.7287
0.5058	2.1067	3000	0.7142
0.4615	2.1770	3100	0.6929

Framework versions

• PEFT 0.10.0
• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1