Hugging Face's logo

Rose-STL-Lab
/
gas-west

PEFT
Safetensors
mistral
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Community
howard commited on
Commit
162b431
•
1 Parent(s): 7454159

for the west coast gas price only

Browse files
Files changed (23) hide show
  1. README.md +42 -42
  2. adapter_config.json +3 -3
  3. adapter_model.bin +1 -1
  4. checkpoint-3178/adapter_config.json +3 -3
  5. checkpoint-3178/adapter_model.safetensors +1 -1
  6. checkpoint-3178/optimizer.pt +1 -1
  7. checkpoint-3178/trainer_state.json +0 -0
  8. checkpoint-3178/training_args.bin +1 -1
  9. checkpoint-3632/adapter_config.json +3 -3
  10. checkpoint-3632/adapter_model.safetensors +1 -1
  11. checkpoint-3632/optimizer.pt +1 -1
  12. checkpoint-3632/trainer_state.json +0 -0
  13. checkpoint-3632/training_args.bin +1 -1
  14. checkpoint-4086/adapter_config.json +3 -3
  15. checkpoint-4086/adapter_model.safetensors +1 -1
  16. checkpoint-4086/optimizer.pt +1 -1
  17. checkpoint-4086/trainer_state.json +0 -0
  18. checkpoint-4086/training_args.bin +1 -1
  19. checkpoint-4540/adapter_config.json +3 -3
  20. checkpoint-4540/adapter_model.safetensors +1 -1
  21. checkpoint-4540/optimizer.pt +1 -1
  22. checkpoint-4540/trainer_state.json +0 -0
  23. checkpoint-4540/training_args.bin +1 -1
README.md CHANGED
@@ -92,12 +92,12 @@ seed: 42
92
 
93
  </details><br>
94
 
95
- [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://rosewandb.ucsd.edu/cht028/finetune/runs/loshr75e)
96
  # finetune/outputs/gas-west
97
 
98
  This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the None dataset.
99
  It achieves the following results on the evaluation set:
100
- - Loss: 0.0002
101
 
102
  ## Model description
103
 
@@ -134,46 +134,46 @@ The following hyperparameters were used during training:
134
 
135
  | Training Loss | Epoch | Step | Validation Loss |
136
  |:-------------:|:------:|:----:|:---------------:|
137
- | 1.4899 | 0.0022 | 1 | 1.3369 |
138
- | 0.6452 | 0.2508 | 114 | 0.6274 |
139
- | 0.3926 | 0.5017 | 228 | 0.4126 |
140
- | 0.1778 | 0.7525 | 342 | 0.2299 |
141
- | 0.0819 | 1.0033 | 456 | 0.1246 |
142
- | 0.0559 | 1.2541 | 570 | 0.0877 |
143
- | 0.0531 | 1.5050 | 684 | 0.0640 |
144
- | 0.0313 | 1.7558 | 798 | 0.0479 |
145
- | 0.0134 | 2.0066 | 912 | 0.0464 |
146
- | 0.0161 | 2.2574 | 1026 | 0.0417 |
147
- | 0.0203 | 2.5083 | 1140 | 0.0361 |
148
- | 0.0143 | 2.7591 | 1254 | 0.0305 |
149
- | 0.0163 | 3.0099 | 1368 | 0.0266 |
150
- | 0.0057 | 3.2607 | 1482 | 0.0180 |
151
- | 0.0087 | 3.5116 | 1596 | 0.0136 |
152
- | 0.0045 | 3.7624 | 1710 | 0.0077 |
153
- | 0.0008 | 4.0132 | 1824 | 0.0052 |
154
- | 0.001 | 4.2640 | 1938 | 0.0032 |
155
- | 0.0002 | 4.5149 | 2052 | 0.0025 |
156
- | 0.0003 | 4.7657 | 2166 | 0.0014 |
157
- | 0.0002 | 5.0165 | 2280 | 0.0009 |
158
- | 0.0003 | 5.2673 | 2394 | 0.0005 |
159
- | 0.0013 | 5.5182 | 2508 | 0.0004 |
160
- | 0.0001 | 5.7690 | 2622 | 0.0003 |
161
- | 0.0001 | 6.0198 | 2736 | 0.0003 |
162
- | 0.0001 | 6.2706 | 2850 | 0.0003 |
163
- | 0.0001 | 6.5215 | 2964 | 0.0003 |
164
- | 0.0001 | 6.7723 | 3078 | 0.0002 |
165
- | 0.0001 | 7.0231 | 3192 | 0.0002 |
166
- | 0.0001 | 7.2739 | 3306 | 0.0002 |
167
- | 0.0001 | 7.5248 | 3420 | 0.0002 |
168
- | 0.0001 | 7.7756 | 3534 | 0.0002 |
169
- | 0.0002 | 8.0264 | 3648 | 0.0002 |
170
- | 0.0002 | 8.2772 | 3762 | 0.0002 |
171
- | 0.0001 | 8.5281 | 3876 | 0.0002 |
172
- | 0.0001 | 8.7789 | 3990 | 0.0002 |
173
- | 0.0001 | 9.0297 | 4104 | 0.0002 |
174
- | 0.0001 | 9.2805 | 4218 | 0.0002 |
175
- | 0.0001 | 9.5314 | 4332 | 0.0002 |
176
- | 0.0001 | 9.7822 | 4446 | 0.0002 |
177
 
178
 
179
  ### Framework versions
 
92
 
93
  </details><br>
94
 
95
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://rosewandb.ucsd.edu/cht028/finetune/runs/5y7pxhrx)
96
  # finetune/outputs/gas-west
97
 
98
  This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the None dataset.
99
  It achieves the following results on the evaluation set:
100
+ - Loss: 0.0003
101
 
102
  ## Model description
103
 
 
134
 
135
  | Training Loss | Epoch | Step | Validation Loss |
136
  |:-------------:|:------:|:----:|:---------------:|
137
+ | 1.4517 | 0.0022 | 1 | 1.3369 |
138
+ | 0.6431 | 0.2508 | 114 | 0.6256 |
139
+ | 0.3998 | 0.5017 | 228 | 0.4131 |
140
+ | 0.1741 | 0.7525 | 342 | 0.2322 |
141
+ | 0.0913 | 1.0033 | 456 | 0.1268 |
142
+ | 0.0679 | 1.2541 | 570 | 0.0809 |
143
+ | 0.0503 | 1.5050 | 684 | 0.0605 |
144
+ | 0.0476 | 1.7558 | 798 | 0.0484 |
145
+ | 0.0084 | 2.0066 | 912 | 0.0417 |
146
+ | 0.0273 | 2.2574 | 1026 | 0.0410 |
147
+ | 0.0296 | 2.5083 | 1140 | 0.0384 |
148
+ | 0.0317 | 2.7591 | 1254 | 0.0344 |
149
+ | 0.0086 | 3.0099 | 1368 | 0.0268 |
150
+ | 0.0076 | 3.2607 | 1482 | 0.0224 |
151
+ | 0.0043 | 3.5116 | 1596 | 0.0206 |
152
+ | 0.0085 | 3.7624 | 1710 | 0.0127 |
153
+ | 0.0071 | 4.0132 | 1824 | 0.0081 |
154
+ | 0.002 | 4.2640 | 1938 | 0.0053 |
155
+ | 0.0028 | 4.5149 | 2052 | 0.0034 |
156
+ | 0.0007 | 4.7657 | 2166 | 0.0016 |
157
+ | 0.0003 | 5.0165 | 2280 | 0.0008 |
158
+ | 0.0002 | 5.2673 | 2394 | 0.0005 |
159
+ | 0.0002 | 5.5182 | 2508 | 0.0004 |
160
+ | 0.0001 | 5.7690 | 2622 | 0.0004 |
161
+ | 0.0001 | 6.0198 | 2736 | 0.0004 |
162
+ | 0.0001 | 6.2706 | 2850 | 0.0004 |
163
+ | 0.0001 | 6.5215 | 2964 | 0.0004 |
164
+ | 0.0001 | 6.7723 | 3078 | 0.0004 |
165
+ | 0.0001 | 7.0231 | 3192 | 0.0004 |
166
+ | 0.0001 | 7.2739 | 3306 | 0.0004 |
167
+ | 0.0001 | 7.5248 | 3420 | 0.0004 |
168
+ | 0.0001 | 7.7756 | 3534 | 0.0004 |
169
+ | 0.0002 | 8.0264 | 3648 | 0.0004 |
170
+ | 0.0002 | 8.2772 | 3762 | 0.0003 |
171
+ | 0.0001 | 8.5281 | 3876 | 0.0004 |
172
+ | 0.0001 | 8.7789 | 3990 | 0.0003 |
173
+ | 0.0002 | 9.0297 | 4104 | 0.0003 |
174
+ | 0.0001 | 9.2805 | 4218 | 0.0003 |
175
+ | 0.0001 | 9.5314 | 4332 | 0.0004 |
176
+ | 0.0001 | 9.7822 | 4446 | 0.0003 |
177
 
178
 
179
  ### Framework versions
adapter_config.json CHANGED
@@ -20,13 +20,13 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "q_proj",
24
  "gate_proj",
 
25
  "k_proj",
 
26
  "o_proj",
27
- "up_proj",
28
  "v_proj",
29
- "down_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
23
  "gate_proj",
24
+ "q_proj",
25
  "k_proj",
26
+ "down_proj",
27
  "o_proj",
 
28
  "v_proj",
29
+ "up_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:69063a3201fae5d5d16adef34ed313027bff72f57c47eabeb89250e070a13d80
3
  size 167934026
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33193aaced75889415645d62a2c9663cd760864ab10f4304a831ddbacb7be301
3
  size 167934026
checkpoint-3178/adapter_config.json CHANGED
@@ -20,13 +20,13 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "q_proj",
24
  "gate_proj",
 
25
  "k_proj",
 
26
  "o_proj",
27
- "up_proj",
28
  "v_proj",
29
- "down_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
23
  "gate_proj",
24
+ "q_proj",
25
  "k_proj",
26
+ "down_proj",
27
  "o_proj",
 
28
  "v_proj",
29
+ "up_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
checkpoint-3178/adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4013d13c51f64a3f73df574ea27f12c53f5bdc1a9737235142f8b3b8f3495076
3
  size 167832688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af75d09941aa410e75084a4d0cb9c274d2efa9f9af4e1a36750ec7a5fe555e97
3
  size 167832688
checkpoint-3178/optimizer.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6aa7d5dbccc5adf34a8c380f6a6c672c359c1a68e1a0c932818d0ca2d9fd65e4
3
  size 671364538
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7f4f8a3ba9b2ee8e28e4a65b9ed1770adb2675aeeb07b3408ee90efd2b22b3d
3
  size 671364538
checkpoint-3178/trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
checkpoint-3178/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:850788f239b860504369af32272e710f256c3d7b981eb329a7799f62be840a95
3
  size 6072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78459f829b6083e305e5286ac1c78edf160f2077290cd2a1eff77d1b67cfdd8b
3
  size 6072
checkpoint-3632/adapter_config.json CHANGED
@@ -20,13 +20,13 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "q_proj",
24
  "gate_proj",
 
25
  "k_proj",
 
26
  "o_proj",
27
- "up_proj",
28
  "v_proj",
29
- "down_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
23
  "gate_proj",
24
+ "q_proj",
25
  "k_proj",
26
+ "down_proj",
27
  "o_proj",
 
28
  "v_proj",
29
+ "up_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
checkpoint-3632/adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:143ecc1e94f31a113063326066bf0481826b908827fd491e41340ce93178e5fc
3
  size 167832688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:862575a4ae141e75178da53f4627bbf333b20fa678f68303fc8b7aa65a67fd69
3
  size 167832688
checkpoint-3632/optimizer.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7eb19ef5f19f3058e1a03e44312198e18fcde3166c2e472cf9f92a270b0b0f69
3
  size 671364538
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac3ba5f37738dd78aab031f633dd27f9914f627f0590934c40e44b69cbac3489
3
  size 671364538
checkpoint-3632/trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
checkpoint-3632/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:850788f239b860504369af32272e710f256c3d7b981eb329a7799f62be840a95
3
  size 6072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78459f829b6083e305e5286ac1c78edf160f2077290cd2a1eff77d1b67cfdd8b
3
  size 6072
checkpoint-4086/adapter_config.json CHANGED
@@ -20,13 +20,13 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "q_proj",
24
  "gate_proj",
 
25
  "k_proj",
 
26
  "o_proj",
27
- "up_proj",
28
  "v_proj",
29
- "down_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
23
  "gate_proj",
24
+ "q_proj",
25
  "k_proj",
26
+ "down_proj",
27
  "o_proj",
 
28
  "v_proj",
29
+ "up_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
checkpoint-4086/adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f491eb7ddb39158b76e6241f1547769b9484624d3f813a331631d855f6db983e
3
  size 167832688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca5b3e2e9a14b122bb2c1fc8481d93134aecdadeed4ba9db53b1701047fd3c52
3
  size 167832688
checkpoint-4086/optimizer.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e5063bb7d52e2645369cbe5f637a27691298639ebf4401b5ed1582ec14fa9c12
3
  size 671364538
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c570406e4674f603d2de0715526d0238b3647f3bd60cfd8b2b87025fb745a912
3
  size 671364538
checkpoint-4086/trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
checkpoint-4086/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:850788f239b860504369af32272e710f256c3d7b981eb329a7799f62be840a95
3
  size 6072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78459f829b6083e305e5286ac1c78edf160f2077290cd2a1eff77d1b67cfdd8b
3
  size 6072
checkpoint-4540/adapter_config.json CHANGED
@@ -20,13 +20,13 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "q_proj",
24
  "gate_proj",
 
25
  "k_proj",
 
26
  "o_proj",
27
- "up_proj",
28
  "v_proj",
29
- "down_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
23
  "gate_proj",
24
+ "q_proj",
25
  "k_proj",
26
+ "down_proj",
27
  "o_proj",
 
28
  "v_proj",
29
+ "up_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
checkpoint-4540/adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a4cc0fbdf0338967fff0af89e96c26b9cad3136c3794bb33bfe598445c01a1f4
3
  size 167832688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1740f20c0c776740397134d55ced0551825a25f672959e3bf584b998d7b78eb
3
  size 167832688
checkpoint-4540/optimizer.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7674f0fa7c3dc271512dd7c0432b315433108ffaeb8873b9fa9b44e15e5408eb
3
  size 671364538
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68615d058768db922830868658dc8092a1a3b57a8dec3af33eb1240695fed591
3
  size 671364538
checkpoint-4540/trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
checkpoint-4540/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:850788f239b860504369af32272e710f256c3d7b981eb329a7799f62be840a95
3
  size 6072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78459f829b6083e305e5286ac1c78edf160f2077290cd2a1eff77d1b67cfdd8b
3
  size 6072