AlekseyKorshuk commited on
Commit
6309bc1
1 Parent(s): 3492c3b

huggingartists

Browse files
README.md CHANGED
@@ -14,11 +14,11 @@ widget:
14
  <div class="inline-flex flex-col" style="line-height: 1.5;">
15
  <div class="flex">
16
  <div
17
- style="display:DISPLAY_1; margin-left: auto; margin-right: auto; width: 92px; height:92px; border-radius: 50%; background-size: cover; background-image: url(&#39;https://images.genius.com/e308b1bc9eeb159ecfa9d807d715f095.1000x1000x1.jpg&#39;)">
18
  </div>
19
  </div>
20
  <div style="text-align: center; margin-top: 3px; font-size: 16px; font-weight: 800">🤖 HuggingArtists Model 🤖</div>
21
- <div style="text-align: center; font-size: 16px; font-weight: 800">SLAVA MARLOW</div>
22
  <a href="https://genius.com/artists/slava-marlow">
23
  <div style="text-align: center; font-size: 14px;">@slava-marlow</div>
24
  </a>
@@ -34,7 +34,7 @@ To understand how the model was developed, check the [W&B report](https://wandb.
34
 
35
  ## Training data
36
 
37
- The model was trained on lyrics from SLAVA MARLOW.
38
 
39
  Dataset is available [here](https://huggingface.co/datasets/huggingartists/slava-marlow).
40
  And can be used with:
@@ -45,15 +45,15 @@ from datasets import load_dataset
45
  dataset = load_dataset("huggingartists/slava-marlow")
46
  ```
47
 
48
- [Explore the data](https://wandb.ai/huggingartists/huggingartists/runs/2poqm17q/artifacts), which is tracked with [W&B artifacts](https://docs.wandb.com/artifacts) at every step of the pipeline.
49
 
50
  ## Training procedure
51
 
52
- The model is based on a pre-trained [GPT-2](https://huggingface.co/gpt2) which is fine-tuned on SLAVA MARLOW's lyrics.
53
 
54
- Hyperparameters and metrics are recorded in the [W&B training run](https://wandb.ai/huggingartists/huggingartists/runs/2panzd8n) for full transparency and reproducibility.
55
 
56
- At the end of training, [the final model](https://wandb.ai/huggingartists/huggingartists/runs/2panzd8n/artifacts) is logged and versioned.
57
 
58
  ## How to use
59
 
 
14
  <div class="inline-flex flex-col" style="line-height: 1.5;">
15
  <div class="flex">
16
  <div
17
+ style="display:DISPLAY_1; margin-left: auto; margin-right: auto; width: 92px; height:92px; border-radius: 50%; background-size: cover; background-image: url(&#39;https://images.genius.com/22e752c5e4e7e4d2e8488babffb33bbf.1000x1000x1.jpg&#39;)">
18
  </div>
19
  </div>
20
  <div style="text-align: center; margin-top: 3px; font-size: 16px; font-weight: 800">🤖 HuggingArtists Model 🤖</div>
21
+ <div style="text-align: center; font-size: 16px; font-weight: 800">MORGENSHTERN</div>
22
  <a href="https://genius.com/artists/slava-marlow">
23
  <div style="text-align: center; font-size: 14px;">@slava-marlow</div>
24
  </a>
 
34
 
35
  ## Training data
36
 
37
+ The model was trained on lyrics from MORGENSHTERN.
38
 
39
  Dataset is available [here](https://huggingface.co/datasets/huggingartists/slava-marlow).
40
  And can be used with:
 
45
  dataset = load_dataset("huggingartists/slava-marlow")
46
  ```
47
 
48
+ [Explore the data](https://wandb.ai/huggingartists/huggingartists/runs/3sgmpuhy/artifacts), which is tracked with [W&B artifacts](https://docs.wandb.com/artifacts) at every step of the pipeline.
49
 
50
  ## Training procedure
51
 
52
+ The model is based on a pre-trained [GPT-2](https://huggingface.co/gpt2) which is fine-tuned on MORGENSHTERN's lyrics.
53
 
54
+ Hyperparameters and metrics are recorded in the [W&B training run](https://wandb.ai/huggingartists/huggingartists/runs/omn4wgyz) for full transparency and reproducibility.
55
 
56
+ At the end of training, [the final model](https://wandb.ai/huggingartists/huggingartists/runs/omn4wgyz/artifacts) is logged and versioned.
57
 
58
  ## How to use
59
 
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "gpt2",
3
  "activation_function": "gelu_new",
4
  "architectures": [
5
  "GPT2LMHeadModel"
 
1
  {
2
+ "_name_or_path": "huggingartists/slava-marlow",
3
  "activation_function": "gelu_new",
4
  "architectures": [
5
  "GPT2LMHeadModel"
flax_model.msgpack CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:003784fb2a244bf7e21fdbbdc3a834eb3efaafb776212f5cf93ec828dbb17488
3
  size 497764120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df1b2b8983b832ed22b122c66ed7bf7c9ff715a4eb640f438af09f67c85dcdb3
3
  size 497764120
optimizer.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d66da2b88a6bf8081f372db1fa0c2cf5cd172181d51df70e580d5f4187cb31c9
3
- size 995603825
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96d52890222d19bf8524cb84e4a332cd839ef36dc1eac420a410a07722ca8431
3
+ size 995604017
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:98afaa2f1ee5ccee595508e8f6e51413def19a720991b4505d84e97c9d97f5d7
3
  size 510403817
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4dbce2307bb9b496425c8f71076a4a1c33b5a768103d6507e3eacaf6b99edcf0
3
  size 510403817
rng_state.pth CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fbc586ea7da20fdcf00ff0887d22ffce1b92144c78c437772b896f540b1d1ffe
3
- size 14567
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13c058b8fa70d384486107f733302e1b72d74991a369b69e1c49457273f7c177
3
+ size 14503
scheduler.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:544ea5472f84fc475d9eea4362936fe8523ee24e1f075c1fe1aa245108e95828
3
  size 623
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3be7b21ee2235eef38a55c0348c96600344a2bc81677cfa58e757a83abbb30f2
3
  size 623
tokenizer_config.json CHANGED
@@ -1 +1 @@
1
- {"unk_token": "<|endoftext|>", "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "model_max_length": 1024, "special_tokens_map_file": null, "name_or_path": "gpt2", "tokenizer_class": "GPT2Tokenizer"}
 
1
+ {"unk_token": "<|endoftext|>", "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "model_max_length": 1024, "special_tokens_map_file": null, "name_or_path": "huggingartists/slava-marlow", "tokenizer_class": "GPT2Tokenizer"}
trainer_state.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "best_metric": null,
3
  "best_model_checkpoint": null,
4
- "epoch": 1.0,
5
- "global_step": 130,
6
  "is_hyper_param_search": false,
7
  "is_local_process_zero": true,
8
  "is_world_process_zero": true,
@@ -162,11 +162,167 @@
162
  "learning_rate": 0.0,
163
  "loss": 1.8023,
164
  "step": 130
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
165
  }
166
  ],
167
- "max_steps": 130,
168
- "num_train_epochs": 1,
169
- "total_flos": 135610564608000.0,
170
  "trial_name": null,
171
  "trial_params": null
172
  }
 
1
  {
2
  "best_metric": null,
3
  "best_model_checkpoint": null,
4
+ "epoch": 2.0,
5
+ "global_step": 260,
6
  "is_hyper_param_search": false,
7
  "is_local_process_zero": true,
8
  "is_world_process_zero": true,
 
162
  "learning_rate": 0.0,
163
  "loss": 1.8023,
164
  "step": 130
165
+ },
166
+ {
167
+ "epoch": 1.04,
168
+ "learning_rate": 5.001712368734975e-07,
169
+ "loss": 1.7351,
170
+ "step": 135
171
+ },
172
+ {
173
+ "epoch": 1.08,
174
+ "learning_rate": 1.9933913245728244e-06,
175
+ "loss": 1.7454,
176
+ "step": 140
177
+ },
178
+ {
179
+ "epoch": 1.12,
180
+ "learning_rate": 4.457885751780542e-06,
181
+ "loss": 1.9051,
182
+ "step": 145
183
+ },
184
+ {
185
+ "epoch": 1.15,
186
+ "learning_rate": 7.857716640189785e-06,
187
+ "loss": 1.818,
188
+ "step": 150
189
+ },
190
+ {
191
+ "epoch": 1.19,
192
+ "learning_rate": 1.2143306799695167e-05,
193
+ "loss": 1.707,
194
+ "step": 155
195
+ },
196
+ {
197
+ "epoch": 1.23,
198
+ "learning_rate": 1.7252162675462473e-05,
199
+ "loss": 1.7507,
200
+ "step": 160
201
+ },
202
+ {
203
+ "epoch": 1.27,
204
+ "learning_rate": 2.3109785644681427e-05,
205
+ "loss": 1.8168,
206
+ "step": 165
207
+ },
208
+ {
209
+ "epoch": 1.31,
210
+ "learning_rate": 2.9630758374242703e-05,
211
+ "loss": 1.7313,
212
+ "step": 170
213
+ },
214
+ {
215
+ "epoch": 1.35,
216
+ "learning_rate": 3.671999039779749e-05,
217
+ "loss": 1.6952,
218
+ "step": 175
219
+ },
220
+ {
221
+ "epoch": 1.38,
222
+ "learning_rate": 4.427410474888204e-05,
223
+ "loss": 1.7249,
224
+ "step": 180
225
+ },
226
+ {
227
+ "epoch": 1.42,
228
+ "learning_rate": 5.218294542987354e-05,
229
+ "loss": 1.6855,
230
+ "step": 185
231
+ },
232
+ {
233
+ "epoch": 1.46,
234
+ "learning_rate": 6.03311837344848e-05,
235
+ "loss": 1.5632,
236
+ "step": 190
237
+ },
238
+ {
239
+ "epoch": 1.5,
240
+ "learning_rate": 6.859999999999999e-05,
241
+ "loss": 1.7108,
242
+ "step": 195
243
+ },
244
+ {
245
+ "epoch": 1.54,
246
+ "learning_rate": 7.686881626551516e-05,
247
+ "loss": 1.816,
248
+ "step": 200
249
+ },
250
+ {
251
+ "epoch": 1.58,
252
+ "learning_rate": 8.501705457012643e-05,
253
+ "loss": 1.8067,
254
+ "step": 205
255
+ },
256
+ {
257
+ "epoch": 1.62,
258
+ "learning_rate": 9.292589525111794e-05,
259
+ "loss": 1.7471,
260
+ "step": 210
261
+ },
262
+ {
263
+ "epoch": 1.65,
264
+ "learning_rate": 0.00010048000960220248,
265
+ "loss": 1.8035,
266
+ "step": 215
267
+ },
268
+ {
269
+ "epoch": 1.69,
270
+ "learning_rate": 0.00010756924162575728,
271
+ "loss": 1.7207,
272
+ "step": 220
273
+ },
274
+ {
275
+ "epoch": 1.73,
276
+ "learning_rate": 0.00011409021435531856,
277
+ "loss": 1.7529,
278
+ "step": 225
279
+ },
280
+ {
281
+ "epoch": 1.77,
282
+ "learning_rate": 0.0001199478373245375,
283
+ "loss": 1.7625,
284
+ "step": 230
285
+ },
286
+ {
287
+ "epoch": 1.81,
288
+ "learning_rate": 0.00012505669320030482,
289
+ "loss": 1.7044,
290
+ "step": 235
291
+ },
292
+ {
293
+ "epoch": 1.85,
294
+ "learning_rate": 0.0001293422833598102,
295
+ "loss": 1.6402,
296
+ "step": 240
297
+ },
298
+ {
299
+ "epoch": 1.88,
300
+ "learning_rate": 0.00013274211424821946,
301
+ "loss": 1.7453,
302
+ "step": 245
303
+ },
304
+ {
305
+ "epoch": 1.92,
306
+ "learning_rate": 0.00013520660867542716,
307
+ "loss": 1.7462,
308
+ "step": 250
309
+ },
310
+ {
311
+ "epoch": 1.96,
312
+ "learning_rate": 0.00013669982876312649,
313
+ "loss": 1.7852,
314
+ "step": 255
315
+ },
316
+ {
317
+ "epoch": 2.0,
318
+ "learning_rate": 0.0001372,
319
+ "loss": 1.7608,
320
+ "step": 260
321
  }
322
  ],
323
+ "max_steps": 260,
324
+ "num_train_epochs": 2,
325
+ "total_flos": 271221129216000.0,
326
  "trial_name": null,
327
  "trial_params": null
328
  }
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:663a0823886f0f29fbe2b3eb1a51d774e2a897437f6b1d5c0c410531a33384eb
3
  size 2671
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5906cf86decfa153fcf84533fd92086ce9d1eb8f6ff2a692ce1232bc92d20c5
3
  size 2671