--- license: mit base_model: prajjwal1/bert-tiny tags: - generated_from_trainer datasets: - sembr2023 metrics: - precision - recall - f1 - accuracy model-index: - name: sembr2023-bert-tiny results: - task: name: Token Classification type: token-classification dataset: name: sembr2023 type: sembr2023 config: sembr2023 split: test args: sembr2023 metrics: - name: Precision type: precision value: 0.7287362872204017 - name: Recall type: recall value: 0.6756042794875181 - name: F1 type: f1 value: 0.7011651816312543 - name: Accuracy type: accuracy value: 0.9490469679439985 --- # sembr2023-bert-tiny This model is a fine-tuned version of [prajjwal1/bert-tiny](https://huggingface.co/prajjwal1/bert-tiny) on the sembr2023 dataset. It achieves the following results on the evaluation set: - Loss: 0.2172 - Precision: 0.7287 - Recall: 0.6756 - F1: 0.7012 - Iou: 0.5398 - Accuracy: 0.9490 - Balanced Accuracy: 0.8256 - Overall Accuracy: 0.9348 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 64 - eval_batch_size: 128 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - training_steps: 1000 ### Training results | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Iou | Accuracy | Balanced Accuracy | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:------:|:--------:|:-----------------:|:----------------:| | 1.422 | 0.07 | 10 | 1.3253 | 0.0 | 0.0 | 0.0 | 0.0 | 0.9114 | 0.4999 | 0.9114 | | 0.8436 | 0.14 | 20 | 0.7947 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.6272 | 0.21 | 30 | 0.5924 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.4967 | 0.28 | 40 | 0.5178 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.4864 | 0.35 | 50 | 0.4818 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.4554 | 0.42 | 60 | 0.4575 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.4635 | 0.49 | 70 | 0.4410 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.4238 | 0.56 | 80 | 0.4261 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.4104 | 0.62 | 90 | 0.4136 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.3832 | 0.69 | 100 | 0.3958 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.3664 | 0.76 | 110 | 0.3693 | 0 | 0.0 | 0.0 | 0.0 | 0.9115 | 0.5 | 0.9115 | | 0.3736 | 0.83 | 120 | 0.3477 | 0.5263 | 0.0013 | 0.0026 | 0.0013 | 0.9115 | 0.5006 | 0.9115 | | 0.3364 | 0.9 | 130 | 0.3317 | 0.5220 | 0.0110 | 0.0215 | 0.0109 | 0.9116 | 0.5050 | 0.9115 | | 0.3095 | 0.97 | 140 | 0.3221 | 0.5584 | 0.0556 | 0.1011 | 0.0533 | 0.9126 | 0.5257 | 0.9118 | | 0.3233 | 1.04 | 150 | 0.3156 | 0.5713 | 0.1471 | 0.2340 | 0.1325 | 0.9148 | 0.5682 | 0.9129 | | 0.3072 | 1.11 | 160 | 0.3112 | 0.6452 | 0.1640 | 0.2616 | 0.1505 | 0.9181 | 0.5776 | 0.9160 | | 0.287 | 1.18 | 170 | 0.3075 | 0.6336 | 0.2622 | 0.3709 | 0.2277 | 0.9213 | 0.6237 | 0.9177 | | 0.3061 | 1.25 | 180 | 0.3039 | 0.7025 | 0.2142 | 0.3283 | 0.1964 | 0.9224 | 0.6027 | 0.9189 | | 0.2924 | 1.32 | 190 | 0.2997 | 0.7581 | 0.1511 | 0.2520 | 0.1442 | 0.9206 | 0.5732 | 0.9183 | | 0.3081 | 1.39 | 200 | 0.2950 | 0.7367 | 0.2025 | 0.3177 | 0.1888 | 0.9230 | 0.5977 | 0.9193 | | 0.2993 | 1.46 | 210 | 0.2921 | 0.6861 | 0.2902 | 0.4079 | 0.2562 | 0.9255 | 0.6387 | 0.9200 | | 0.275 | 1.53 | 220 | 0.2885 | 0.6734 | 0.3249 | 0.4383 | 0.2807 | 0.9263 | 0.6548 | 0.9200 | | 0.2692 | 1.6 | 230 | 0.2861 | 0.6622 | 0.3438 | 0.4526 | 0.2925 | 0.9264 | 0.6634 | 0.9188 | | 0.2536 | 1.67 | 240 | 0.2828 | 0.6295 | 0.3895 | 0.4812 | 0.3169 | 0.9257 | 0.6836 | 0.9176 | | 0.265 | 1.74 | 250 | 0.2790 | 0.6586 | 0.3546 | 0.4610 | 0.2996 | 0.9266 | 0.6684 | 0.9200 | | 0.2571 | 1.81 | 260 | 0.2736 | 0.6641 | 0.3729 | 0.4776 | 0.3137 | 0.9278 | 0.6773 | 0.9208 | | 0.2684 | 1.88 | 270 | 0.2711 | 0.6794 | 0.4142 | 0.5146 | 0.3465 | 0.9309 | 0.6976 | 0.9222 | | 0.2754 | 1.94 | 280 | 0.2721 | 0.6408 | 0.4813 | 0.5497 | 0.3790 | 0.9302 | 0.7276 | 0.9198 | | 0.2507 | 2.01 | 290 | 0.2652 | 0.6922 | 0.4438 | 0.5408 | 0.3707 | 0.9333 | 0.7123 | 0.9240 | | 0.2678 | 2.08 | 300 | 0.2622 | 0.6957 | 0.4243 | 0.5271 | 0.3578 | 0.9326 | 0.7031 | 0.9244 | | 0.2676 | 2.15 | 310 | 0.2629 | 0.6698 | 0.5020 | 0.5739 | 0.4024 | 0.9340 | 0.7390 | 0.9228 | | 0.2369 | 2.22 | 320 | 0.2596 | 0.6667 | 0.5123 | 0.5794 | 0.4079 | 0.9342 | 0.7437 | 0.9233 | | 0.2293 | 2.29 | 330 | 0.2560 | 0.6741 | 0.5174 | 0.5854 | 0.4138 | 0.9352 | 0.7465 | 0.9240 | | 0.235 | 2.36 | 340 | 0.2517 | 0.7160 | 0.4796 | 0.5744 | 0.4030 | 0.9371 | 0.7306 | 0.9270 | | 0.209 | 2.43 | 350 | 0.2545 | 0.6704 | 0.5332 | 0.5940 | 0.4225 | 0.9355 | 0.7539 | 0.9236 | | 0.2032 | 2.5 | 360 | 0.2486 | 0.7002 | 0.5130 | 0.5922 | 0.4206 | 0.9375 | 0.7458 | 0.9267 | | 0.2005 | 2.57 | 370 | 0.2482 | 0.6870 | 0.5418 | 0.6058 | 0.4345 | 0.9376 | 0.7589 | 0.9259 | | 0.206 | 2.64 | 380 | 0.2463 | 0.6949 | 0.5384 | 0.6067 | 0.4354 | 0.9382 | 0.7577 | 0.9264 | | 0.2196 | 2.71 | 390 | 0.2421 | 0.7158 | 0.5237 | 0.6049 | 0.4336 | 0.9395 | 0.7518 | 0.9289 | | 0.1863 | 2.78 | 400 | 0.2410 | 0.7072 | 0.5435 | 0.6146 | 0.4437 | 0.9397 | 0.7608 | 0.9282 | | 0.2036 | 2.85 | 410 | 0.2408 | 0.6889 | 0.5775 | 0.6283 | 0.4580 | 0.9395 | 0.7761 | 0.9272 | | 0.1982 | 2.92 | 420 | 0.2345 | 0.7198 | 0.5596 | 0.6297 | 0.4595 | 0.9418 | 0.7692 | 0.9300 | | 0.1909 | 2.99 | 430 | 0.2327 | 0.7173 | 0.5830 | 0.6432 | 0.4741 | 0.9428 | 0.7804 | 0.9307 | | 0.2286 | 3.06 | 440 | 0.2309 | 0.7250 | 0.5933 | 0.6526 | 0.4843 | 0.9441 | 0.7857 | 0.9317 | | 0.1839 | 3.12 | 450 | 0.2305 | 0.7219 | 0.6159 | 0.6647 | 0.4978 | 0.9450 | 0.7964 | 0.9323 | | 0.2086 | 3.19 | 460 | 0.2280 | 0.7252 | 0.6171 | 0.6668 | 0.5002 | 0.9454 | 0.7972 | 0.9328 | | 0.2055 | 3.26 | 470 | 0.2302 | 0.7088 | 0.6450 | 0.6754 | 0.5099 | 0.9451 | 0.8096 | 0.9318 | | 0.1925 | 3.33 | 480 | 0.2252 | 0.7309 | 0.6263 | 0.6746 | 0.5090 | 0.9465 | 0.8020 | 0.9336 | | 0.165 | 3.4 | 490 | 0.2248 | 0.7254 | 0.6364 | 0.6780 | 0.5128 | 0.9465 | 0.8065 | 0.9336 | | 0.1814 | 3.47 | 500 | 0.2283 | 0.7008 | 0.6637 | 0.6818 | 0.5172 | 0.9452 | 0.8181 | 0.9314 | | 0.1812 | 3.54 | 510 | 0.2239 | 0.7275 | 0.6436 | 0.6830 | 0.5186 | 0.9471 | 0.8101 | 0.9336 | | 0.1738 | 3.61 | 520 | 0.2237 | 0.7241 | 0.6498 | 0.6850 | 0.5209 | 0.9471 | 0.8129 | 0.9335 | | 0.1726 | 3.68 | 530 | 0.2227 | 0.7271 | 0.6517 | 0.6873 | 0.5236 | 0.9475 | 0.8140 | 0.9338 | | 0.188 | 3.75 | 540 | 0.2204 | 0.7407 | 0.6393 | 0.6863 | 0.5224 | 0.9483 | 0.8088 | 0.9348 | | 0.187 | 3.82 | 550 | 0.2185 | 0.7539 | 0.6303 | 0.6866 | 0.5227 | 0.9491 | 0.8052 | 0.9362 | | 0.1917 | 3.89 | 560 | 0.2193 | 0.7354 | 0.6532 | 0.6919 | 0.5289 | 0.9485 | 0.8152 | 0.9349 | | 0.1794 | 3.96 | 570 | 0.2197 | 0.7326 | 0.6574 | 0.6929 | 0.5301 | 0.9485 | 0.8170 | 0.9346 | | 0.1541 | 4.03 | 580 | 0.2203 | 0.7292 | 0.6645 | 0.6954 | 0.5330 | 0.9485 | 0.8203 | 0.9343 | | 0.1837 | 4.1 | 590 | 0.2190 | 0.7336 | 0.6563 | 0.6928 | 0.5300 | 0.9485 | 0.8166 | 0.9349 | | 0.1541 | 4.17 | 600 | 0.2177 | 0.7405 | 0.6467 | 0.6904 | 0.5272 | 0.9487 | 0.8123 | 0.9354 | | 0.1721 | 4.24 | 610 | 0.2210 | 0.7178 | 0.6767 | 0.6966 | 0.5345 | 0.9479 | 0.8254 | 0.9338 | | 0.1657 | 4.31 | 620 | 0.2186 | 0.7323 | 0.6628 | 0.6958 | 0.5335 | 0.9487 | 0.8196 | 0.9350 | | 0.1792 | 4.38 | 630 | 0.2182 | 0.7294 | 0.6668 | 0.6967 | 0.5345 | 0.9486 | 0.8214 | 0.9349 | | 0.1908 | 4.44 | 640 | 0.2183 | 0.7309 | 0.6648 | 0.6963 | 0.5341 | 0.9487 | 0.8205 | 0.9348 | | 0.1581 | 4.51 | 650 | 0.2177 | 0.7330 | 0.6658 | 0.6978 | 0.5359 | 0.9490 | 0.8211 | 0.9349 | | 0.169 | 4.58 | 660 | 0.2178 | 0.7313 | 0.6685 | 0.6985 | 0.5366 | 0.9489 | 0.8223 | 0.9347 | | 0.1756 | 4.65 | 670 | 0.2184 | 0.7271 | 0.6723 | 0.6986 | 0.5369 | 0.9487 | 0.8239 | 0.9344 | | 0.1563 | 4.72 | 680 | 0.2179 | 0.7311 | 0.6706 | 0.6996 | 0.5379 | 0.9490 | 0.8233 | 0.9349 | | 0.1684 | 4.79 | 690 | 0.2161 | 0.7475 | 0.6565 | 0.6990 | 0.5373 | 0.9500 | 0.8175 | 0.9362 | | 0.1585 | 4.86 | 700 | 0.2171 | 0.7380 | 0.6648 | 0.6995 | 0.5378 | 0.9495 | 0.8209 | 0.9354 | | 0.1841 | 4.93 | 710 | 0.2181 | 0.7283 | 0.6745 | 0.7004 | 0.5389 | 0.9489 | 0.8251 | 0.9346 | | 0.1724 | 5.0 | 720 | 0.2177 | 0.7323 | 0.6695 | 0.6995 | 0.5379 | 0.9491 | 0.8229 | 0.9349 | | 0.1791 | 5.07 | 730 | 0.2170 | 0.7329 | 0.6708 | 0.7005 | 0.5391 | 0.9492 | 0.8236 | 0.9351 | | 0.1712 | 5.14 | 740 | 0.2171 | 0.7344 | 0.6705 | 0.7010 | 0.5396 | 0.9494 | 0.8235 | 0.9354 | | 0.1489 | 5.21 | 750 | 0.2164 | 0.7374 | 0.6683 | 0.7012 | 0.5398 | 0.9496 | 0.8226 | 0.9357 | | 0.157 | 5.28 | 760 | 0.2161 | 0.7407 | 0.6636 | 0.7000 | 0.5385 | 0.9497 | 0.8205 | 0.9358 | | 0.1686 | 5.35 | 770 | 0.2180 | 0.7262 | 0.6775 | 0.7010 | 0.5396 | 0.9489 | 0.8263 | 0.9345 | | 0.1526 | 5.42 | 780 | 0.2168 | 0.7344 | 0.6690 | 0.7002 | 0.5387 | 0.9493 | 0.8228 | 0.9351 | | 0.1542 | 5.49 | 790 | 0.2172 | 0.7313 | 0.6722 | 0.7005 | 0.5390 | 0.9491 | 0.8241 | 0.9349 | | 0.1498 | 5.56 | 800 | 0.2168 | 0.7351 | 0.6691 | 0.7005 | 0.5391 | 0.9494 | 0.8229 | 0.9353 | | 0.1571 | 5.62 | 810 | 0.2167 | 0.7348 | 0.6687 | 0.7002 | 0.5387 | 0.9493 | 0.8227 | 0.9354 | | 0.1682 | 5.69 | 820 | 0.2175 | 0.7265 | 0.6775 | 0.7011 | 0.5398 | 0.9489 | 0.8263 | 0.9347 | | 0.1688 | 5.76 | 830 | 0.2175 | 0.7267 | 0.6764 | 0.7006 | 0.5392 | 0.9489 | 0.8259 | 0.9347 | | 0.1622 | 5.83 | 840 | 0.2161 | 0.7393 | 0.6633 | 0.6992 | 0.5376 | 0.9495 | 0.8203 | 0.9357 | | 0.1547 | 5.9 | 850 | 0.2173 | 0.7282 | 0.6755 | 0.7008 | 0.5395 | 0.9490 | 0.8255 | 0.9347 | | 0.1712 | 5.97 | 860 | 0.2166 | 0.7339 | 0.6701 | 0.7005 | 0.5391 | 0.9493 | 0.8232 | 0.9352 | | 0.1632 | 6.04 | 870 | 0.2168 | 0.7317 | 0.6724 | 0.7008 | 0.5394 | 0.9492 | 0.8242 | 0.9352 | | 0.1615 | 6.11 | 880 | 0.2167 | 0.7315 | 0.6727 | 0.7009 | 0.5395 | 0.9492 | 0.8244 | 0.9352 | | 0.1543 | 6.18 | 890 | 0.2164 | 0.7348 | 0.6699 | 0.7008 | 0.5395 | 0.9494 | 0.8232 | 0.9354 | | 0.1407 | 6.25 | 900 | 0.2168 | 0.7318 | 0.6726 | 0.7009 | 0.5396 | 0.9492 | 0.8243 | 0.9351 | | 0.1607 | 6.32 | 910 | 0.2170 | 0.7299 | 0.6743 | 0.7010 | 0.5396 | 0.9491 | 0.8250 | 0.9350 | | 0.1652 | 6.39 | 920 | 0.2172 | 0.7276 | 0.6760 | 0.7009 | 0.5395 | 0.9489 | 0.8257 | 0.9347 | | 0.1676 | 6.46 | 930 | 0.2173 | 0.7274 | 0.6765 | 0.7010 | 0.5397 | 0.9489 | 0.8260 | 0.9347 | | 0.14 | 6.53 | 940 | 0.2174 | 0.7267 | 0.6767 | 0.7008 | 0.5394 | 0.9489 | 0.8260 | 0.9346 | | 0.1634 | 6.6 | 950 | 0.2173 | 0.7276 | 0.6764 | 0.7011 | 0.5397 | 0.9490 | 0.8259 | 0.9347 | | 0.174 | 6.67 | 960 | 0.2172 | 0.7283 | 0.6759 | 0.7011 | 0.5398 | 0.9490 | 0.8257 | 0.9348 | | 0.156 | 6.74 | 970 | 0.2172 | 0.7287 | 0.6759 | 0.7013 | 0.5400 | 0.9491 | 0.8257 | 0.9348 | | 0.1641 | 6.81 | 980 | 0.2172 | 0.7287 | 0.6756 | 0.7012 | 0.5398 | 0.9490 | 0.8256 | 0.9348 | | 0.1634 | 6.88 | 990 | 0.2172 | 0.7287 | 0.6756 | 0.7012 | 0.5398 | 0.9490 | 0.8256 | 0.9348 | | 0.1753 | 6.94 | 1000 | 0.2172 | 0.7287 | 0.6756 | 0.7012 | 0.5398 | 0.9490 | 0.8256 | 0.9348 | ### Framework versions - Transformers 4.34.1 - Pytorch 2.0.1 - Datasets 2.14.6 - Tokenizers 0.14.1