Hugging Face's logo

tsavage68
/
UTI_L3_1000steps_1e5rate_SFT

Text Generation
Transformers
Safetensors
llama
trl
sft
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
UTI_L3_1000steps_1e5rate_SFT / README.md
tsavage68's picture
tsavage68
End of training
b24dd81 verified
preview code
|
raw
history blame contribute delete
3.54 kB
metadata
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: UTI_L3_1000steps_1e5rate_SFT
    results: []

UTI_L3_1000steps_1e5rate_SFT

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.5151

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 2
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 100
  • training_steps: 1000

Training results

Training Loss Epoch Step Validation Loss
1.6211 0.3333 25 1.3130
1.3098 0.6667 50 1.3151
1.3699 1.0 75 1.3342
1.1671 1.3333 100 1.3720
1.1955 1.6667 125 1.4161
1.178 2.0 150 1.3563
0.7205 2.3333 175 1.4594
0.7783 2.6667 200 1.4538
0.7533 3.0 225 1.4397
0.3678 3.3333 250 1.6859
0.3873 3.6667 275 1.6593
0.4191 4.0 300 1.7873
0.1775 4.3333 325 1.9132
0.1884 4.6667 350 1.8814
0.1946 5.0 375 2.0113
0.1227 5.3333 400 2.0399
0.1261 5.6667 425 2.1039
0.1223 6.0 450 2.1222
0.0939 6.3333 475 2.1375
0.0994 6.6667 500 2.1088
0.1026 7.0 525 2.1071
0.0803 7.3333 550 2.2376
0.0792 7.6667 575 2.2282
0.0824 8.0 600 2.2099
0.0656 8.3333 625 2.2770
0.0625 8.6667 650 2.3860
0.0698 9.0 675 2.3219
0.0489 9.3333 700 2.3820
0.0521 9.6667 725 2.4133
0.0478 10.0 750 2.4260
0.0442 10.3333 775 2.4633
0.0464 10.6667 800 2.4853
0.0484 11.0 825 2.4851
0.0422 11.3333 850 2.5013
0.0425 11.6667 875 2.5089
0.0436 12.0 900 2.5127
0.0427 12.3333 925 2.5145
0.041 12.6667 950 2.5150
0.0413 13.0 975 2.5156
0.0412 13.3333 1000 2.5151

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.0.0+cu117
  • Datasets 2.19.2
  • Tokenizers 0.19.1