File size: 9,965 Bytes
9008d50 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 |
Visible device: cuda
Seed used: 0
Batch size: 64
Epochs: 40
Learning rate: 1e-05
Entropy weight: 0.01
Regularization weight: 0.0
Only use multiwoz like domains: False
We use: 1.0% of the data
Dialogue order used: 0
Vectorizer: Data set used is multiwoz21
We filter state by active domains: True
Vectorizer: Data set used is multiwoz21
Embedding semantic descriptions: True
Embedded descriptions successfully. Size: torch.Size([338, 768])
Data set used for descriptions: multiwoz21
We use Roberta to embed actions.
Loaded model from experiments/seed0/save/supervised.pol.mdl
Start training
Epoch: 0
Average actions: 1.9973957538604736
Average target actions: 2.5520834922790527
Precision: 0.09615384615384616
Recall: 0.07462686567164178
F1: 0.08403361344537816
<<dialog policy>> epoch 0: saved network to mdl
Best Precision: 0.09615384615384616
Best Recall: 0.07462686567164178
Best F1: 0.08403361344537816
Epoch: 1
Precision: 0.09615384615384616
Recall: 0.07462686567164178
F1: 0.08403361344537816
Best Precision: 0.09615384615384616
Best Recall: 0.07462686567164178
Best F1: 0.08403361344537816
Epoch: 2
Average actions: 2.3515625
Average target actions: 2.6197917461395264
Precision: 0.10526315789473684
Recall: 0.08955223880597014
F1: 0.0967741935483871
<<dialog policy>> epoch 2: saved network to mdl
Best Precision: 0.10526315789473684
Best Recall: 0.08955223880597014
Best F1: 0.0967741935483871
Epoch: 3
Precision: 0.10526315789473684
Recall: 0.08955223880597014
F1: 0.0967741935483871
Best Precision: 0.10526315789473684
Best Recall: 0.08955223880597014
Best F1: 0.0967741935483871
Epoch: 4
Average actions: 1.6770832538604736
Average target actions: 2.8567709922790527
Precision: 0.1347517730496454
Recall: 0.0945273631840796
F1: 0.11111111111111112
<<dialog policy>> epoch 4: saved network to mdl
Best Precision: 0.1347517730496454
Best Recall: 0.0945273631840796
Best F1: 0.11111111111111112
Epoch: 5
Precision: 0.1347517730496454
Recall: 0.0945273631840796
F1: 0.11111111111111112
Best Precision: 0.1347517730496454
Best Recall: 0.0945273631840796
Best F1: 0.11111111111111112
Epoch: 6
Average actions: 1.9088542461395264
Average target actions: 2.7213542461395264
Precision: 0.12080536912751678
Recall: 0.08955223880597014
F1: 0.10285714285714286
Best Precision: 0.1347517730496454
Best Recall: 0.0945273631840796
Best F1: 0.11111111111111112
Epoch: 7
Precision: 0.12080536912751678
Recall: 0.08955223880597014
F1: 0.10285714285714286
Best Precision: 0.1347517730496454
Best Recall: 0.0945273631840796
Best F1: 0.11111111111111112
Epoch: 8
Average actions: 2.0572915077209473
Average target actions: 2.8229167461395264
Precision: 0.12903225806451613
Recall: 0.09950248756218906
F1: 0.11235955056179776
<<dialog policy>> epoch 8: saved network to mdl
Best Precision: 0.1347517730496454
Best Recall: 0.09950248756218906
Best F1: 0.11235955056179776
Epoch: 9
Precision: 0.12903225806451613
Recall: 0.09950248756218906
F1: 0.11235955056179776
Best Precision: 0.1347517730496454
Best Recall: 0.09950248756218906
Best F1: 0.11235955056179776
Epoch: 10
Average actions: 2.0911459922790527
Average target actions: 2.6875
Precision: 0.11612903225806452
Recall: 0.08955223880597014
F1: 0.10112359550561797
Best Precision: 0.1347517730496454
Best Recall: 0.09950248756218906
Best F1: 0.11235955056179776
Epoch: 11
Precision: 0.11612903225806452
Recall: 0.08955223880597014
F1: 0.10112359550561797
Best Precision: 0.1347517730496454
Best Recall: 0.09950248756218906
Best F1: 0.11235955056179776
Epoch: 12
Average actions: 2.0833332538604736
Average target actions: 2.5859375
Precision: 0.11976047904191617
Recall: 0.09950248756218906
F1: 0.10869565217391305
Best Precision: 0.1347517730496454
Best Recall: 0.09950248756218906
Best F1: 0.11235955056179776
Epoch: 13
Precision: 0.11976047904191617
Recall: 0.09950248756218906
F1: 0.10869565217391305
Best Precision: 0.1347517730496454
Best Recall: 0.09950248756218906
Best F1: 0.11235955056179776
Epoch: 14
Average actions: 2.1119790077209473
Average target actions: 2.7213542461395264
Precision: 0.16778523489932887
Recall: 0.12437810945273632
F1: 0.14285714285714285
<<dialog policy>> epoch 14: saved network to mdl
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 15
Precision: 0.16778523489932887
Recall: 0.12437810945273632
F1: 0.14285714285714285
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 16
Average actions: 1.7994792461395264
Average target actions: 2.5520834922790527
Precision: 0.10135135135135136
Recall: 0.07462686567164178
F1: 0.08595988538681948
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 17
Precision: 0.10135135135135136
Recall: 0.07462686567164178
F1: 0.08595988538681948
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 18
Average actions: 2.0572915077209473
Average target actions: 2.7552084922790527
Precision: 0.13548387096774195
Recall: 0.1044776119402985
F1: 0.11797752808988765
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 19
Precision: 0.13548387096774195
Recall: 0.1044776119402985
F1: 0.11797752808988765
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 20
Average actions: 1.9661457538604736
Average target actions: 2.7213542461395264
Precision: 0.1118421052631579
Recall: 0.0845771144278607
F1: 0.0963172804532578
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 21
Precision: 0.1118421052631579
Recall: 0.0845771144278607
F1: 0.0963172804532578
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 22
Average actions: 1.9557292461395264
Average target actions: 2.5520834922790527
Precision: 0.07741935483870968
Recall: 0.05970149253731343
F1: 0.06741573033707865
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 23
Precision: 0.07741935483870968
Recall: 0.05970149253731343
F1: 0.06741573033707865
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 24
Average actions: 2.0833334922790527
Average target actions: 2.8229167461395264
Precision: 0.09090909090909091
Recall: 0.06965174129353234
F1: 0.07887323943661972
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 25
Precision: 0.09090909090909091
Recall: 0.06965174129353234
F1: 0.07887323943661972
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 26
Average actions: 1.7135417461395264
Average target actions: 2.6197917461395264
Precision: 0.145985401459854
Recall: 0.09950248756218906
F1: 0.1183431952662722
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 27
Precision: 0.145985401459854
Recall: 0.09950248756218906
F1: 0.1183431952662722
Best Precision: 0.16778523489932887
Best Recall: 0.12437810945273632
Best F1: 0.14285714285714285
Epoch: 28
Average actions: 2.0364584922790527
Average target actions: 2.5520834922790527
Precision: 0.16891891891891891
Recall: 0.12437810945273632
F1: 0.14326647564469916
<<dialog policy>> epoch 28: saved network to mdl
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 29
Precision: 0.16891891891891891
Recall: 0.12437810945273632
F1: 0.14326647564469916
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 30
Average actions: 2.0026040077209473
Average target actions: 2.3828125
Precision: 0.16216216216216217
Recall: 0.11940298507462686
F1: 0.13753581661891118
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 31
Precision: 0.16216216216216217
Recall: 0.11940298507462686
F1: 0.13753581661891118
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 32
Average actions: 1.8046875
Average target actions: 2.6875
Precision: 0.12142857142857143
Recall: 0.0845771144278607
F1: 0.09970674486803519
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 33
Precision: 0.12142857142857143
Recall: 0.0845771144278607
F1: 0.09970674486803519
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 34
Average actions: 1.9348957538604736
Average target actions: 2.6875
Precision: 0.12162162162162163
Recall: 0.08955223880597014
F1: 0.10315186246418337
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 35
Precision: 0.12162162162162163
Recall: 0.08955223880597014
F1: 0.10315186246418337
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 36
Average actions: 2.0989584922790527
Average target actions: 2.484375
Precision: 0.14743589743589744
Recall: 0.11442786069651742
F1: 0.1288515406162465
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 37
Precision: 0.14743589743589744
Recall: 0.11442786069651742
F1: 0.1288515406162465
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 38
Average actions: 2.0260415077209473
Average target actions: 2.5520834922790527
Precision: 0.1456953642384106
Recall: 0.10945273631840796
F1: 0.12499999999999997
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
Epoch: 39
Precision: 0.1456953642384106
Recall: 0.10945273631840796
F1: 0.12499999999999997
Best Precision: 0.16891891891891891
Best Recall: 0.12437810945273632
Best F1: 0.14326647564469916
|