Hugging Face's logo

anikethjr
/
PromoGen_K562_GPT2_4096_tokens_2080Ti_x4_more_DE

Text Generation
Transformers
PyTorch
TensorBoard
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
anikethjr commited on
Commit
0d61a56
•
1 Parent(s): e1efe43

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +175 -0
README.md ADDED
@@ -0,0 +1,175 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: PromoGen_K562_GPT2_4096_tokens_2080Ti_x4_more_DE
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # PromoGen_K562_GPT2_4096_tokens_2080Ti_x4_more_DE
13
+
14
+ This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 10.7123
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 0.001
36
+ - train_batch_size: 128
37
+ - eval_batch_size: 128
38
+ - seed: 42
39
+ - gradient_accumulation_steps: 8
40
+ - total_train_batch_size: 1024
41
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
+ - lr_scheduler_type: cosine
43
+ - lr_scheduler_warmup_steps: 1000
44
+ - num_epochs: 200
45
+
46
+ ### Training results
47
+
48
+ | Training Loss | Epoch | Step | Validation Loss |
49
+ |:-------------:|:------:|:-----:|:---------------:|
50
+ | 7.8346 | 1.69 | 250 | 7.3201 |
51
+ | 6.8483 | 3.38 | 500 | 6.6602 |
52
+ | 6.5149 | 5.07 | 750 | 6.3857 |
53
+ | 6.1941 | 6.76 | 1000 | 6.1764 |
54
+ | 6.0188 | 8.45 | 1250 | 6.0308 |
55
+ | 5.7091 | 10.14 | 1500 | 5.9934 |
56
+ | 5.4459 | 11.82 | 1750 | 5.9371 |
57
+ | 5.1559 | 13.51 | 2000 | 5.9561 |
58
+ | 5.053 | 15.2 | 2250 | 6.0799 |
59
+ | 4.8356 | 16.89 | 2500 | 5.9637 |
60
+ | 4.6625 | 18.58 | 2750 | 6.0990 |
61
+ | 4.4349 | 20.27 | 3000 | 6.4003 |
62
+ | 4.2347 | 21.96 | 3250 | 6.2304 |
63
+ | 4.0581 | 23.65 | 3500 | 6.4391 |
64
+ | 4.1369 | 25.34 | 3750 | 6.6182 |
65
+ | 3.9847 | 27.03 | 4000 | 6.8092 |
66
+ | 3.7499 | 28.72 | 4250 | 6.7152 |
67
+ | 3.6274 | 30.41 | 4500 | 6.8939 |
68
+ | 3.515 | 32.09 | 4750 | 7.1118 |
69
+ | 3.428 | 33.78 | 5000 | 6.8671 |
70
+ | 3.4504 | 35.47 | 5250 | 7.0974 |
71
+ | 3.3346 | 37.16 | 5500 | 7.4627 |
72
+ | 3.1574 | 38.85 | 5750 | 7.1213 |
73
+ | 3.0594 | 40.54 | 6000 | 7.3364 |
74
+ | 3.1296 | 42.23 | 6250 | 7.5662 |
75
+ | 3.0683 | 43.92 | 6500 | 7.3176 |
76
+ | 2.9588 | 45.61 | 6750 | 7.4911 |
77
+ | 2.8816 | 47.3 | 7000 | 7.7823 |
78
+ | 2.7364 | 48.99 | 7250 | 7.4892 |
79
+ | 2.6647 | 50.68 | 7500 | 7.7526 |
80
+ | 2.8314 | 52.36 | 7750 | 7.9454 |
81
+ | 2.7594 | 54.05 | 8000 | 8.2020 |
82
+ | 2.5894 | 55.74 | 8250 | 7.8243 |
83
+ | 2.5204 | 57.43 | 8500 | 8.0420 |
84
+ | 2.5072 | 59.12 | 8750 | 8.2496 |
85
+ | 2.4765 | 60.81 | 9000 | 7.9321 |
86
+ | 2.5039 | 62.5 | 9250 | 8.1675 |
87
+ | 2.4301 | 64.19 | 9500 | 8.4031 |
88
+ | 2.2991 | 65.88 | 9750 | 8.0673 |
89
+ | 2.2471 | 67.57 | 10000 | 8.2341 |
90
+ | 2.3514 | 69.26 | 10250 | 8.5114 |
91
+ | 2.3166 | 70.95 | 10500 | 8.1837 |
92
+ | 2.2225 | 72.63 | 10750 | 8.3777 |
93
+ | 2.17 | 74.32 | 11000 | 8.5542 |
94
+ | 2.0509 | 76.01 | 11250 | 8.6312 |
95
+ | 2.0204 | 77.7 | 11500 | 8.4680 |
96
+ | 2.1625 | 79.39 | 11750 | 8.7197 |
97
+ | 2.1102 | 81.08 | 12000 | 8.8730 |
98
+ | 1.9783 | 82.77 | 12250 | 8.6112 |
99
+ | 1.9293 | 84.46 | 12500 | 8.7798 |
100
+ | 1.9402 | 86.15 | 12750 | 8.9528 |
101
+ | 1.9095 | 87.84 | 13000 | 8.6741 |
102
+ | 1.9188 | 89.53 | 13250 | 8.9483 |
103
+ | 1.877 | 91.22 | 13500 | 9.0630 |
104
+ | 1.7614 | 92.9 | 13750 | 8.8313 |
105
+ | 1.7251 | 94.59 | 14000 | 8.9837 |
106
+ | 1.821 | 96.28 | 14250 | 9.1340 |
107
+ | 1.799 | 97.97 | 14500 | 8.9597 |
108
+ | 1.7024 | 99.66 | 14750 | 9.1133 |
109
+ | 1.6657 | 101.35 | 15000 | 9.2959 |
110
+ | 1.5863 | 103.04 | 15250 | 9.2909 |
111
+ | 1.5708 | 104.73 | 15500 | 9.1864 |
112
+ | 1.6538 | 106.42 | 15750 | 9.3570 |
113
+ | 1.6119 | 108.11 | 16000 | 9.4937 |
114
+ | 1.5147 | 109.8 | 16250 | 9.3262 |
115
+ | 1.4745 | 111.49 | 16500 | 9.4693 |
116
+ | 1.4869 | 113.18 | 16750 | 9.5963 |
117
+ | 1.4664 | 114.86 | 17000 | 9.4436 |
118
+ | 1.4566 | 116.55 | 17250 | 9.5559 |
119
+ | 1.4231 | 118.24 | 17500 | 9.6734 |
120
+ | 1.3421 | 119.93 | 17750 | 9.5746 |
121
+ | 1.3086 | 121.62 | 18000 | 9.7034 |
122
+ | 1.3773 | 123.31 | 18250 | 9.7930 |
123
+ | 1.3537 | 125.0 | 18500 | 9.6953 |
124
+ | 1.2834 | 126.69 | 18750 | 9.8154 |
125
+ | 1.2516 | 128.38 | 19000 | 9.8966 |
126
+ | 1.2079 | 130.07 | 19250 | 9.9296 |
127
+ | 1.1875 | 131.76 | 19500 | 9.9139 |
128
+ | 1.2227 | 133.45 | 19750 | 10.0024 |
129
+ | 1.1899 | 135.14 | 20000 | 10.0485 |
130
+ | 1.1332 | 136.82 | 20250 | 10.0329 |
131
+ | 1.107 | 138.51 | 20500 | 10.1044 |
132
+ | 1.1128 | 140.2 | 20750 | 10.1534 |
133
+ | 1.0945 | 141.89 | 21000 | 10.1375 |
134
+ | 1.0762 | 143.58 | 21250 | 10.2039 |
135
+ | 1.0495 | 145.27 | 21500 | 10.2378 |
136
+ | 1.0105 | 146.96 | 21750 | 10.2377 |
137
+ | 0.9875 | 148.65 | 22000 | 10.3128 |
138
+ | 1.0177 | 150.34 | 22250 | 10.3502 |
139
+ | 0.9956 | 152.03 | 22500 | 10.3521 |
140
+ | 0.9628 | 153.72 | 22750 | 10.3924 |
141
+ | 0.9403 | 155.41 | 23000 | 10.4238 |
142
+ | 0.9214 | 157.09 | 23250 | 10.4514 |
143
+ | 0.9078 | 158.78 | 23500 | 10.4660 |
144
+ | 0.9114 | 160.47 | 23750 | 10.4969 |
145
+ | 0.8935 | 162.16 | 24000 | 10.5225 |
146
+ | 0.8711 | 163.85 | 24250 | 10.5399 |
147
+ | 0.8549 | 165.54 | 24500 | 10.5598 |
148
+ | 0.8564 | 167.23 | 24750 | 10.5896 |
149
+ | 0.8441 | 168.92 | 25000 | 10.5900 |
150
+ | 0.8357 | 170.61 | 25250 | 10.6085 |
151
+ | 0.8229 | 172.3 | 25500 | 10.6296 |
152
+ | 0.8094 | 173.99 | 25750 | 10.6426 |
153
+ | 0.7992 | 175.68 | 26000 | 10.6518 |
154
+ | 0.8038 | 177.36 | 26250 | 10.6652 |
155
+ | 0.7948 | 179.05 | 26500 | 10.6742 |
156
+ | 0.7852 | 180.74 | 26750 | 10.6780 |
157
+ | 0.7775 | 182.43 | 27000 | 10.6912 |
158
+ | 0.7737 | 184.12 | 27250 | 10.6925 |
159
+ | 0.769 | 185.81 | 27500 | 10.6993 |
160
+ | 0.7679 | 187.5 | 27750 | 10.7020 |
161
+ | 0.7636 | 189.19 | 28000 | 10.7062 |
162
+ | 0.76 | 190.88 | 28250 | 10.7085 |
163
+ | 0.7595 | 192.57 | 28500 | 10.7107 |
164
+ | 0.7583 | 194.26 | 28750 | 10.7120 |
165
+ | 0.7552 | 195.95 | 29000 | 10.7126 |
166
+ | 0.7568 | 197.63 | 29250 | 10.7120 |
167
+ | 0.7556 | 199.32 | 29500 | 10.7123 |
168
+
169
+
170
+ ### Framework versions
171
+
172
+ - Transformers 4.24.0
173
+ - Pytorch 1.13.0
174
+ - Datasets 2.7.0
175
+ - Tokenizers 0.13.0.dev0