Edit model card

results

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4476

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 2
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 17
  • training_steps: 1792

Training results

Training Loss Epoch Step Validation Loss
2.8156 0.1115 50 1.7682
1.6754 0.2230 100 1.6347
1.5257 0.3344 150 1.5543
1.5435 0.4459 200 1.5301
1.613 0.5574 250 1.5579
1.6178 0.6689 300 1.6006
1.675 0.7804 350 1.5209
1.5046 0.8919 400 1.5203
1.5977 1.0033 450 1.5253
1.5303 1.1148 500 1.4984
1.4748 1.2263 550 1.5073
1.4955 1.3378 600 1.4998
1.5737 1.4493 650 1.5405
1.5662 1.5608 700 1.5147
1.417 1.6722 750 1.5047
1.5732 1.7837 800 1.4768
1.5077 1.8952 850 1.4948
1.5634 2.0067 900 1.4768
1.5219 2.1182 950 1.4752
1.4073 2.2297 1000 1.4776
1.4915 2.3411 1050 1.4799
1.4585 2.4526 1100 1.4868
1.4979 2.5641 1150 1.4703
1.4755 2.6756 1200 1.4617
1.4167 2.7871 1250 1.4576
1.5095 2.8986 1300 1.4651
1.477 3.0100 1350 1.4526
1.4495 3.1215 1400 1.4660
1.5609 3.2330 1450 1.4547
1.4319 3.3445 1500 1.4478
1.3469 3.4560 1550 1.4597
1.4454 3.5674 1600 1.4478
1.4071 3.6789 1650 1.4506
1.3686 3.7904 1700 1.4485
1.4958 3.9019 1750 1.4476

Framework versions

  • PEFT 0.13.0
  • Transformers 4.44.2
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.19.1
Downloads last month
2
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for Orcawise/custom_mistral_7b_orcawise_sep_27

Adapter
(1171)
this model