Hugging Face's logo

tsavage68
/
UTI_M2_1000steps_1e6rate_SFT

Text Generation
Transformers
Safetensors
mistral
trl
sft
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
UTI_M2_1000steps_1e6rate_SFT / README.md
tsavage68's picture
tsavage68
End of training
58d6d5b verified
preview code
|
raw
history blame contribute delete
3.54 kB
metadata
license: apache-2.0
base_model: mistralai/Mistral-7B-Instruct-v0.2
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: UTI_M2_1000steps_1e6rate_SFT
    results: []

UTI_M2_1000steps_1e6rate_SFT

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.7960

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-06
  • train_batch_size: 2
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 100
  • training_steps: 1000

Training results

Training Loss Epoch Step Validation Loss
2.2167 0.3333 25 1.1865
0.9806 0.6667 50 0.9618
0.936 1.0 75 0.9371
0.8294 1.3333 100 0.9512
0.8273 1.6667 125 0.9369
0.7851 2.0 150 0.9036
0.5263 2.3333 175 0.9990
0.5512 2.6667 200 0.9589
0.5272 3.0 225 0.9576
0.2888 3.3333 250 1.1371
0.2968 3.6667 275 1.1164
0.3381 4.0 300 1.1144
0.1802 4.3333 325 1.1697
0.2025 4.6667 350 1.1946
0.2273 5.0 375 1.2614
0.1417 5.3333 400 1.3260
0.1524 5.6667 425 1.3343
0.136 6.0 450 1.3735
0.117 6.3333 475 1.3843
0.1284 6.6667 500 1.3742
0.1172 7.0 525 1.4114
0.0905 7.3333 550 1.5000
0.1027 7.6667 575 1.5142
0.097 8.0 600 1.4912
0.0837 8.3333 625 1.5974
0.0832 8.6667 650 1.6185
0.0781 9.0 675 1.6203
0.0698 9.3333 700 1.6833
0.0722 9.6667 725 1.6960
0.0681 10.0 750 1.7139
0.0635 10.3333 775 1.7732
0.0654 10.6667 800 1.7704
0.0663 11.0 825 1.7647
0.0604 11.3333 850 1.7840
0.0628 11.6667 875 1.7916
0.0627 12.0 900 1.7947
0.061 12.3333 925 1.7962
0.062 12.6667 950 1.7967
0.0607 13.0 975 1.7960
0.0605 13.3333 1000 1.7960

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.0.0+cu117
  • Datasets 2.19.2
  • Tokenizers 0.19.1