Hugging Face's logo

tsavage68
/
UTI_L3_1000steps_1e7rate_SFT

Text Generation
Transformers
Safetensors
llama
trl
sft
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
tsavage68 commited on
Commit
ff37341
•
1 Parent(s): e33fdfa

End of training

Browse files
Files changed (15) hide show
  1. README.md +43 -43
  2. config.json +1 -1
  3. final_checkpoint/config.json +1 -1
  4. final_checkpoint/generation_config.json +1 -1
  5. final_checkpoint/model-00001-of-00004.safetensors +1 -1
  6. final_checkpoint/model-00002-of-00004.safetensors +1 -1
  7. final_checkpoint/model-00003-of-00004.safetensors +1 -1
  8. final_checkpoint/model-00004-of-00004.safetensors +1 -1
  9. generation_config.json +1 -1
  10. model-00001-of-00004.safetensors +1 -1
  11. model-00002-of-00004.safetensors +1 -1
  12. model-00003-of-00004.safetensors +1 -1
  13. model-00004-of-00004.safetensors +1 -1
  14. tokenizer.json +1 -1
  15. training_args.bin +1 -1
README.md CHANGED
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 2.4527
21
 
22
  ## Model description
23
 
@@ -51,51 +51,51 @@ The following hyperparameters were used during training:
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-------:|:----:|:---------------:|
54
- | 2.7452 | 0.3333 | 25 | 2.7748 |
55
- | 2.7844 | 0.6667 | 50 | 2.7704 |
56
- | 2.7915 | 1.0 | 75 | 2.7596 |
57
- | 2.7945 | 1.3333 | 100 | 2.7379 |
58
- | 2.6611 | 1.6667 | 125 | 2.7124 |
59
- | 2.5528 | 2.0 | 150 | 2.6876 |
60
- | 2.6652 | 2.3333 | 175 | 2.6628 |
61
- | 2.6808 | 2.6667 | 200 | 2.6394 |
62
- | 2.668 | 3.0 | 225 | 2.6175 |
63
- | 2.5973 | 3.3333 | 250 | 2.5970 |
64
- | 2.4943 | 3.6667 | 275 | 2.5785 |
65
- | 2.5433 | 4.0 | 300 | 2.5620 |
66
- | 2.6383 | 4.3333 | 325 | 2.5468 |
67
- | 2.5221 | 4.6667 | 350 | 2.5333 |
68
- | 2.5698 | 5.0 | 375 | 2.5210 |
69
- | 2.5026 | 5.3333 | 400 | 2.5108 |
70
- | 2.5267 | 5.6667 | 425 | 2.5004 |
71
- | 2.4484 | 6.0 | 450 | 2.4920 |
72
- | 2.4735 | 6.3333 | 475 | 2.4844 |
73
- | 2.3763 | 6.6667 | 500 | 2.4780 |
74
- | 2.5461 | 7.0 | 525 | 2.4729 |
75
- | 2.5406 | 7.3333 | 550 | 2.4691 |
76
- | 2.4936 | 7.6667 | 575 | 2.4645 |
77
- | 2.4328 | 8.0 | 600 | 2.4615 |
78
- | 2.4954 | 8.3333 | 625 | 2.4590 |
79
- | 2.4458 | 8.6667 | 650 | 2.4564 |
80
- | 2.5661 | 9.0 | 675 | 2.4550 |
81
- | 2.4158 | 9.3333 | 700 | 2.4542 |
82
- | 2.4964 | 9.6667 | 725 | 2.4537 |
83
- | 2.5488 | 10.0 | 750 | 2.4530 |
84
- | 2.4364 | 10.3333 | 775 | 2.4530 |
85
- | 2.3929 | 10.6667 | 800 | 2.4520 |
86
- | 2.536 | 11.0 | 825 | 2.4528 |
87
- | 2.5173 | 11.3333 | 850 | 2.4526 |
88
- | 2.4415 | 11.6667 | 875 | 2.4524 |
89
- | 2.5111 | 12.0 | 900 | 2.4522 |
90
- | 2.4223 | 12.3333 | 925 | 2.4528 |
91
- | 2.4031 | 12.6667 | 950 | 2.4527 |
92
- | 2.4848 | 13.0 | 975 | 2.4527 |
93
- | 2.4349 | 13.3333 | 1000 | 2.4527 |
94
 
95
 
96
  ### Framework versions
97
 
98
- - Transformers 4.41.1
99
  - Pytorch 2.0.0+cu117
100
- - Datasets 2.19.1
101
  - Tokenizers 0.19.1
 
17
 
18
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.6055
21
 
22
  ## Model description
23
 
 
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-------:|:----:|:---------------:|
54
+ | 2.4485 | 0.3333 | 25 | 2.4666 |
55
+ | 2.4645 | 0.6667 | 50 | 2.4522 |
56
+ | 2.452 | 1.0 | 75 | 2.4164 |
57
+ | 2.391 | 1.3333 | 100 | 2.3529 |
58
+ | 2.2816 | 1.6667 | 125 | 2.2866 |
59
+ | 2.175 | 2.0 | 150 | 2.2255 |
60
+ | 2.2168 | 2.3333 | 175 | 2.1683 |
61
+ | 2.1574 | 2.6667 | 200 | 2.1166 |
62
+ | 2.1107 | 3.0 | 225 | 2.0679 |
63
+ | 2.0126 | 3.3333 | 250 | 2.0229 |
64
+ | 1.9353 | 3.6667 | 275 | 1.9810 |
65
+ | 1.9552 | 4.0 | 300 | 1.9445 |
66
+ | 1.9759 | 4.3333 | 325 | 1.9100 |
67
+ | 1.8721 | 4.6667 | 350 | 1.8773 |
68
+ | 1.8928 | 5.0 | 375 | 1.8491 |
69
+ | 1.8331 | 5.3333 | 400 | 1.8236 |
70
+ | 1.8221 | 5.6667 | 425 | 1.7980 |
71
+ | 1.7615 | 6.0 | 450 | 1.7762 |
72
+ | 1.7701 | 6.3333 | 475 | 1.7562 |
73
+ | 1.7034 | 6.6667 | 500 | 1.7327 |
74
+ | 1.7471 | 7.0 | 525 | 1.7064 |
75
+ | 1.7317 | 7.3333 | 550 | 1.6831 |
76
+ | 1.6897 | 7.6667 | 575 | 1.6645 |
77
+ | 1.6452 | 8.0 | 600 | 1.6476 |
78
+ | 1.6675 | 8.3333 | 625 | 1.6327 |
79
+ | 1.569 | 8.6667 | 650 | 1.6238 |
80
+ | 1.705 | 9.0 | 675 | 1.6163 |
81
+ | 1.6025 | 9.3333 | 700 | 1.6121 |
82
+ | 1.6224 | 9.6667 | 725 | 1.6083 |
83
+ | 1.6976 | 10.0 | 750 | 1.6074 |
84
+ | 1.6031 | 10.3333 | 775 | 1.6059 |
85
+ | 1.5703 | 10.6667 | 800 | 1.6046 |
86
+ | 1.6563 | 11.0 | 825 | 1.6055 |
87
+ | 1.6464 | 11.3333 | 850 | 1.6059 |
88
+ | 1.6075 | 11.6667 | 875 | 1.6055 |
89
+ | 1.6453 | 12.0 | 900 | 1.6057 |
90
+ | 1.5754 | 12.3333 | 925 | 1.6054 |
91
+ | 1.5962 | 12.6667 | 950 | 1.6055 |
92
+ | 1.6333 | 13.0 | 975 | 1.6055 |
93
+ | 1.6086 | 13.3333 | 1000 | 1.6055 |
94
 
95
 
96
  ### Framework versions
97
 
98
+ - Transformers 4.41.2
99
  - Pytorch 2.0.0+cu117
100
+ - Datasets 2.19.2
101
  - Tokenizers 0.19.1
config.json CHANGED
@@ -23,7 +23,7 @@
23
  "rope_theta": 500000.0,
24
  "tie_word_embeddings": false,
25
  "torch_dtype": "float16",
26
- "transformers_version": "4.41.1",
27
  "use_cache": false,
28
  "vocab_size": 128256
29
  }
 
23
  "rope_theta": 500000.0,
24
  "tie_word_embeddings": false,
25
  "torch_dtype": "float16",
26
+ "transformers_version": "4.41.2",
27
  "use_cache": false,
28
  "vocab_size": 128256
29
  }
final_checkpoint/config.json CHANGED
@@ -23,7 +23,7 @@
23
  "rope_theta": 500000.0,
24
  "tie_word_embeddings": false,
25
  "torch_dtype": "float16",
26
- "transformers_version": "4.41.1",
27
  "use_cache": false,
28
  "vocab_size": 128256
29
  }
 
23
  "rope_theta": 500000.0,
24
  "tie_word_embeddings": false,
25
  "torch_dtype": "float16",
26
+ "transformers_version": "4.41.2",
27
  "use_cache": false,
28
  "vocab_size": 128256
29
  }
final_checkpoint/generation_config.json CHANGED
@@ -8,5 +8,5 @@
8
  "max_length": 4096,
9
  "temperature": 0.6,
10
  "top_p": 0.9,
11
- "transformers_version": "4.41.1"
12
  }
 
8
  "max_length": 4096,
9
  "temperature": 0.6,
10
  "top_p": 0.9,
11
+ "transformers_version": "4.41.2"
12
  }
final_checkpoint/model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de02372fd73778f94cb22aaa5d36a19fc59c63df0b5e265f6a967e3ef67d53ff
3
  size 4976698592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0aa8f9daf2a398aaffc1fe86488894d30002a2ec16e3296a468a28aa2e4f7c4
3
  size 4976698592
final_checkpoint/model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:466c281810a37d3739ba16110746dbf54a611c75e04dac1b908086c3d4be1873
3
  size 4999802616
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a116f5f2d2e8b3db2185069bdccb1cd0931e72fd5ab848484df6c35198b4183
3
  size 4999802616
final_checkpoint/model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9a7a4324ac13665afe367be2bd6c4c02698936f094124d9a3db855f71893e24
3
  size 4915916080
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1bdd653468c11abc2e5e958a30fe08553ead1585dd66d322a0b75ad5fe8906d5
3
  size 4915916080
final_checkpoint/model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f26db4ce2a22a214d06f7b9c0b028bea622f6325b068e71aad03e26b4a554589
3
  size 1168138808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56b9a072fd07f9d6759e65680be2e0b27be4aad25944640931af34d88f6123c4
3
  size 1168138808
generation_config.json CHANGED
@@ -8,5 +8,5 @@
8
  "max_length": 4096,
9
  "temperature": 0.6,
10
  "top_p": 0.9,
11
- "transformers_version": "4.41.1"
12
  }
 
8
  "max_length": 4096,
9
  "temperature": 0.6,
10
  "top_p": 0.9,
11
+ "transformers_version": "4.41.2"
12
  }
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de02372fd73778f94cb22aaa5d36a19fc59c63df0b5e265f6a967e3ef67d53ff
3
  size 4976698592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0aa8f9daf2a398aaffc1fe86488894d30002a2ec16e3296a468a28aa2e4f7c4
3
  size 4976698592
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:466c281810a37d3739ba16110746dbf54a611c75e04dac1b908086c3d4be1873
3
  size 4999802616
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a116f5f2d2e8b3db2185069bdccb1cd0931e72fd5ab848484df6c35198b4183
3
  size 4999802616
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9a7a4324ac13665afe367be2bd6c4c02698936f094124d9a3db855f71893e24
3
  size 4915916080
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1bdd653468c11abc2e5e958a30fe08553ead1585dd66d322a0b75ad5fe8906d5
3
  size 4915916080
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f26db4ce2a22a214d06f7b9c0b028bea622f6325b068e71aad03e26b4a554589
3
  size 1168138808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56b9a072fd07f9d6759e65680be2e0b27be4aad25944640931af34d88f6123c4
3
  size 1168138808
tokenizer.json CHANGED
@@ -2,7 +2,7 @@
2
  "version": "1.0",
3
  "truncation": {
4
  "direction": "Right",
5
- "max_length": 100,
6
  "strategy": "LongestFirst",
7
  "stride": 0
8
  },
 
2
  "version": "1.0",
3
  "truncation": {
4
  "direction": "Right",
5
+ "max_length": 1024,
6
  "strategy": "LongestFirst",
7
  "stride": 0
8
  },
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:70aae47f07b7e57e1c7985416109771df0233f6b0c7386d8ebec5a26102ed0bf
3
  size 4603
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a4150ef7d0dff8eb90f52565dda56ec2a77189da81959a1199bbd9cec0632a8
3
  size 4603