--- license: mit base_model: gpt2 tags: - generated_from_trainer model-index: - name: 8_bar_lmd_clean_custom_final_token_scheme_epochs10 results: [] --- # 8_bar_lmd_clean_custom_final_token_scheme_epochs10 This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 5.0474 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.005 - train_batch_size: 48 - eval_batch_size: 32 - seed: 1 - gradient_accumulation_steps: 2 - total_train_batch_size: 96 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - lr_scheduler_warmup_ratio: 0.01 - num_epochs: 5 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | 8.2004 | 0.03 | 10 | 5.8581 | | 5.8284 | 0.06 | 20 | 5.7712 | | 5.7438 | 0.09 | 30 | 5.7220 | | 5.682 | 0.11 | 40 | 5.6185 | | 5.6048 | 0.14 | 50 | 5.5303 | | 5.5522 | 0.17 | 60 | 5.5000 | | 5.5071 | 0.2 | 70 | 5.4540 | | 5.4512 | 0.23 | 80 | 5.3570 | | 5.2932 | 0.26 | 90 | 5.1704 | | 5.1931 | 0.29 | 100 | 5.1101 | | 5.109 | 0.32 | 110 | 5.0844 | | 5.1019 | 0.34 | 120 | 5.0580 | | 5.0702 | 0.37 | 130 | 5.0452 | | 5.0727 | 0.4 | 140 | 5.0375 | | 5.0654 | 0.43 | 150 | 5.0376 | | 5.0625 | 0.46 | 160 | 5.0292 | | 5.0426 | 0.49 | 170 | 5.0376 | | 5.0428 | 0.52 | 180 | 5.0328 | | 5.0153 | 0.54 | 190 | 5.0280 | | 5.0307 | 0.57 | 200 | 5.0256 | | 5.0321 | 0.6 | 210 | 5.0202 | | 5.0638 | 0.63 | 220 | 5.0150 | | 5.0363 | 0.66 | 230 | 5.0077 | | 5.0151 | 0.69 | 240 | 5.0067 | | 5.0359 | 0.72 | 250 | 5.0131 | | 4.9987 | 0.74 | 260 | 4.9949 | | 5.0128 | 0.77 | 270 | 4.9826 | | 5.0164 | 0.8 | 280 | 4.9872 | | 4.9941 | 0.83 | 290 | 4.9869 | | 5.0258 | 0.86 | 300 | 4.9968 | | 5.0132 | 0.89 | 310 | 5.0012 | | 5.0568 | 0.92 | 320 | 5.0877 | | 5.1161 | 0.95 | 330 | 5.0727 | | 5.1321 | 0.97 | 340 | 5.1333 | | 5.1533 | 1.0 | 350 | 5.1079 | | 5.1469 | 1.03 | 360 | 5.0910 | | 5.1172 | 1.06 | 370 | 5.0900 | | 5.1259 | 1.09 | 380 | 5.1383 | | 5.1742 | 1.12 | 390 | 5.1814 | | 5.2056 | 1.15 | 400 | 5.1766 | | 5.1964 | 1.17 | 410 | 5.1935 | | 5.2422 | 1.2 | 420 | 5.2032 | | 5.2544 | 1.23 | 430 | 5.1723 | | 5.2853 | 1.26 | 440 | 5.2317 | | 5.302 | 1.29 | 450 | 5.3312 | | 5.319 | 1.32 | 460 | 5.2069 | | 5.2633 | 1.35 | 470 | 5.2114 | | 5.2548 | 1.38 | 480 | 5.2350 | | 5.294 | 1.4 | 490 | 5.3470 | | 5.2876 | 1.43 | 500 | 5.1773 | | 5.2857 | 1.46 | 510 | 5.2445 | | 5.3095 | 1.49 | 520 | 5.2099 | | 5.2322 | 1.52 | 530 | 5.2158 | | 5.215 | 1.55 | 540 | 5.1505 | | 5.2248 | 1.58 | 550 | 5.1520 | | 5.2123 | 1.6 | 560 | 5.1412 | | 5.2098 | 1.63 | 570 | 5.1431 | | 5.2088 | 1.66 | 580 | 5.1443 | | 5.2007 | 1.69 | 590 | 5.1595 | | 5.212 | 1.72 | 600 | 5.2016 | | 5.2143 | 1.75 | 610 | 5.1499 | | 5.2152 | 1.78 | 620 | 5.1333 | | 5.2003 | 1.81 | 630 | 5.1810 | | 5.2761 | 1.83 | 640 | 5.1993 | | 5.2707 | 1.86 | 650 | 5.1884 | | 5.2622 | 1.89 | 660 | 5.1815 | | 5.242 | 1.92 | 670 | 5.1830 | | 5.2705 | 1.95 | 680 | 5.2060 | | 5.28 | 1.98 | 690 | 5.1905 | | 5.2443 | 2.01 | 700 | 5.1681 | | 5.2252 | 2.03 | 710 | 5.1609 | | 5.2256 | 2.06 | 720 | 5.1565 | | 5.2043 | 2.09 | 730 | 5.1541 | | 5.2366 | 2.12 | 740 | 5.1640 | | 5.2441 | 2.15 | 750 | 5.1742 | | 5.2585 | 2.18 | 760 | 5.1817 | | 5.2182 | 2.21 | 770 | 5.1588 | | 5.2153 | 2.23 | 780 | 5.1710 | | 5.218 | 2.26 | 790 | 5.1414 | | 5.2067 | 2.29 | 800 | 5.1374 | | 5.1897 | 2.32 | 810 | 5.1294 | | 5.195 | 2.35 | 820 | 5.1315 | | 5.2114 | 2.38 | 830 | 5.1335 | | 5.2061 | 2.41 | 840 | 5.1366 | | 5.1933 | 2.44 | 850 | 5.1307 | | 5.1856 | 2.46 | 860 | 5.1311 | | 5.1972 | 2.49 | 870 | 5.1317 | | 5.1958 | 2.52 | 880 | 5.1356 | | 5.2131 | 2.55 | 890 | 5.1300 | | 5.1906 | 2.58 | 900 | 5.1177 | | 5.1984 | 2.61 | 910 | 5.1166 | | 5.1799 | 2.64 | 920 | 5.1297 | | 5.1874 | 2.66 | 930 | 5.1328 | | 5.1896 | 2.69 | 940 | 5.1219 | | 5.1864 | 2.72 | 950 | 5.1286 | | 5.203 | 2.75 | 960 | 5.1238 | | 5.2038 | 2.78 | 970 | 5.1183 | | 5.1992 | 2.81 | 980 | 5.1116 | | 5.1733 | 2.84 | 990 | 5.1099 | | 5.1604 | 2.87 | 1000 | 5.1101 | | 5.1657 | 2.89 | 1010 | 5.1067 | | 5.1594 | 2.92 | 1020 | 5.0977 | | 5.1501 | 2.95 | 1030 | 5.0945 | | 5.1485 | 2.98 | 1040 | 5.0897 | | 5.164 | 3.01 | 1050 | 5.0835 | | 5.1394 | 3.04 | 1060 | 5.0955 | | 5.1472 | 3.07 | 1070 | 5.1000 | | 5.2015 | 3.09 | 1080 | 5.0947 | | 5.1665 | 3.12 | 1090 | 5.0916 | | 5.1483 | 3.15 | 1100 | 5.0958 | | 5.1753 | 3.18 | 1110 | 5.0902 | | 5.1565 | 3.21 | 1120 | 5.0886 | | 5.1572 | 3.24 | 1130 | 5.0876 | | 5.1193 | 3.27 | 1140 | 5.0900 | | 5.144 | 3.3 | 1150 | 5.0807 | | 5.1362 | 3.32 | 1160 | 5.0913 | | 5.1574 | 3.35 | 1170 | 5.0780 | | 5.1381 | 3.38 | 1180 | 5.0738 | | 5.1405 | 3.41 | 1190 | 5.0739 | | 5.1463 | 3.44 | 1200 | 5.0739 | | 5.1324 | 3.47 | 1210 | 5.0729 | | 5.1102 | 3.5 | 1220 | 5.0703 | | 5.1575 | 3.52 | 1230 | 5.0700 | | 5.125 | 3.55 | 1240 | 5.0674 | | 5.1391 | 3.58 | 1250 | 5.0673 | | 5.1405 | 3.61 | 1260 | 5.0678 | | 5.141 | 3.64 | 1270 | 5.0708 | | 5.1059 | 3.67 | 1280 | 5.0719 | | 5.1423 | 3.7 | 1290 | 5.0719 | | 5.1098 | 3.72 | 1300 | 5.0698 | | 5.1165 | 3.75 | 1310 | 5.0674 | | 5.1249 | 3.78 | 1320 | 5.0660 | | 5.1129 | 3.81 | 1330 | 5.0675 | | 5.1469 | 3.84 | 1340 | 5.0677 | | 5.1215 | 3.87 | 1350 | 5.0688 | | 5.1392 | 3.9 | 1360 | 5.0685 | | 5.1355 | 3.93 | 1370 | 5.0674 | | 5.1372 | 3.95 | 1380 | 5.0666 | | 5.1237 | 3.98 | 1390 | 5.0641 | | 5.1452 | 4.01 | 1400 | 5.0623 | | 5.117 | 4.04 | 1410 | 5.0642 | | 5.1467 | 4.07 | 1420 | 5.0604 | | 5.1221 | 4.1 | 1430 | 5.0590 | | 5.0959 | 4.13 | 1440 | 5.0540 | | 5.1088 | 4.15 | 1450 | 5.0538 | | 5.116 | 4.18 | 1460 | 5.0542 | | 5.1293 | 4.21 | 1470 | 5.0542 | | 5.1337 | 4.24 | 1480 | 5.0526 | | 5.1154 | 4.27 | 1490 | 5.0522 | | 5.1196 | 4.3 | 1500 | 5.0538 | | 5.1122 | 4.33 | 1510 | 5.0515 | | 5.094 | 4.36 | 1520 | 5.0503 | | 5.116 | 4.38 | 1530 | 5.0505 | | 5.1142 | 4.41 | 1540 | 5.0520 | | 5.1106 | 4.44 | 1550 | 5.0517 | | 5.1023 | 4.47 | 1560 | 5.0502 | | 5.1153 | 4.5 | 1570 | 5.0497 | | 5.1096 | 4.53 | 1580 | 5.0493 | | 5.1331 | 4.56 | 1590 | 5.0503 | | 5.1178 | 4.58 | 1600 | 5.0506 | | 5.0984 | 4.61 | 1610 | 5.0503 | | 5.0992 | 4.64 | 1620 | 5.0493 | | 5.0888 | 4.67 | 1630 | 5.0487 | | 5.1153 | 4.7 | 1640 | 5.0484 | | 5.1102 | 4.73 | 1650 | 5.0480 | | 5.1143 | 4.76 | 1660 | 5.0478 | | 5.1012 | 4.79 | 1670 | 5.0477 | | 5.104 | 4.81 | 1680 | 5.0478 | | 5.1129 | 4.84 | 1690 | 5.0475 | | 5.0908 | 4.87 | 1700 | 5.0475 | | 5.091 | 4.9 | 1710 | 5.0474 | | 5.1115 | 4.93 | 1720 | 5.0474 | | 5.1267 | 4.96 | 1730 | 5.0474 | | 5.1245 | 4.99 | 1740 | 5.0474 | ### Framework versions - Transformers 4.36.0 - Pytorch 2.1.0 - Datasets 2.15.0 - Tokenizers 0.15.1