--- license: apache-2.0 base_model: distilbert-base-multilingual-cased tags: - generated_from_trainer metrics: - precision - recall - f1 - accuracy model-index: - name: sembr2023-distilbert-base-multilingual-cased results: [] --- # sembr2023-distilbert-base-multilingual-cased This model is a fine-tuned version of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.2282 - Precision: 0.7986 - Recall: 0.8244 - F1: 0.8113 - Iou: 0.6825 - Accuracy: 0.9666 - Balanced Accuracy: 0.9023 - Overall Accuracy: 0.9521 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 64 - eval_batch_size: 128 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - training_steps: 1000 ### Training results | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Iou | Accuracy | Balanced Accuracy | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:------:|:--------:|:-----------------:|:----------------:| | 0.4021 | 0.06 | 10 | 0.3642 | 0 | 0.0 | 0.0 | 0.0 | 0.9130 | 0.5 | 0.9130 | | 0.2804 | 0.12 | 20 | 0.2550 | 0.7462 | 0.5589 | 0.6391 | 0.4696 | 0.9451 | 0.7704 | 0.9331 | | 0.2173 | 0.18 | 30 | 0.2038 | 0.9157 | 0.4420 | 0.5962 | 0.4247 | 0.9479 | 0.7191 | 0.9406 | | 0.1831 | 0.24 | 40 | 0.1824 | 0.8317 | 0.7186 | 0.7710 | 0.6274 | 0.9629 | 0.8524 | 0.9455 | | 0.1816 | 0.29 | 50 | 0.1829 | 0.7540 | 0.7857 | 0.7695 | 0.6254 | 0.9591 | 0.8807 | 0.9405 | | 0.1461 | 0.35 | 60 | 0.1619 | 0.8520 | 0.7210 | 0.7810 | 0.6407 | 0.9648 | 0.8545 | 0.9498 | | 0.1424 | 0.41 | 70 | 0.1568 | 0.8091 | 0.7717 | 0.7900 | 0.6528 | 0.9643 | 0.8772 | 0.9483 | | 0.11 | 0.47 | 80 | 0.1550 | 0.8189 | 0.7760 | 0.7969 | 0.6624 | 0.9656 | 0.8798 | 0.9504 | | 0.1373 | 0.53 | 90 | 0.1584 | 0.8248 | 0.7640 | 0.7933 | 0.6573 | 0.9654 | 0.8743 | 0.9484 | | 0.1202 | 0.59 | 100 | 0.1612 | 0.8292 | 0.7608 | 0.7935 | 0.6577 | 0.9656 | 0.8729 | 0.9496 | | 0.0966 | 0.65 | 110 | 0.1390 | 0.8575 | 0.7597 | 0.8057 | 0.6746 | 0.9681 | 0.8738 | 0.9551 | | 0.0832 | 0.71 | 120 | 0.1569 | 0.8292 | 0.7858 | 0.8069 | 0.6763 | 0.9673 | 0.8852 | 0.9508 | | 0.0914 | 0.76 | 130 | 0.1399 | 0.8295 | 0.8009 | 0.8150 | 0.6877 | 0.9684 | 0.8926 | 0.9529 | | 0.0793 | 0.82 | 140 | 0.1456 | 0.8161 | 0.7914 | 0.8035 | 0.6716 | 0.9663 | 0.8872 | 0.9512 | | 0.0903 | 0.88 | 150 | 0.1461 | 0.8218 | 0.7882 | 0.8047 | 0.6732 | 0.9667 | 0.8860 | 0.9523 | | 0.0976 | 0.94 | 160 | 0.1380 | 0.7999 | 0.8304 | 0.8148 | 0.6875 | 0.9672 | 0.9053 | 0.9533 | | 0.0776 | 1.0 | 170 | 0.1633 | 0.8053 | 0.8144 | 0.8099 | 0.6805 | 0.9667 | 0.8978 | 0.9492 | | 0.0722 | 1.06 | 180 | 0.1581 | 0.8569 | 0.7739 | 0.8133 | 0.6853 | 0.9691 | 0.8808 | 0.9528 | | 0.0787 | 1.12 | 190 | 0.1563 | 0.7780 | 0.8350 | 0.8055 | 0.6743 | 0.9649 | 0.9062 | 0.9495 | | 0.0642 | 1.18 | 200 | 0.1483 | 0.8007 | 0.8115 | 0.8060 | 0.6751 | 0.9660 | 0.8961 | 0.9527 | | 0.0582 | 1.24 | 210 | 0.1559 | 0.8244 | 0.8067 | 0.8154 | 0.6884 | 0.9682 | 0.8952 | 0.9534 | | 0.0617 | 1.29 | 220 | 0.1359 | 0.8158 | 0.8115 | 0.8136 | 0.6858 | 0.9677 | 0.8970 | 0.9560 | | 0.0508 | 1.35 | 230 | 0.1638 | 0.8372 | 0.7872 | 0.8114 | 0.6827 | 0.9682 | 0.8863 | 0.9548 | | 0.049 | 1.41 | 240 | 0.1699 | 0.8585 | 0.7432 | 0.7967 | 0.6620 | 0.9670 | 0.8657 | 0.9523 | | 0.0604 | 1.47 | 250 | 0.1452 | 0.7935 | 0.8374 | 0.8148 | 0.6875 | 0.9669 | 0.9083 | 0.9523 | | 0.0512 | 1.53 | 260 | 0.1551 | 0.8353 | 0.7820 | 0.8078 | 0.6776 | 0.9676 | 0.8837 | 0.9542 | | 0.0496 | 1.59 | 270 | 0.1686 | 0.7757 | 0.8490 | 0.8107 | 0.6816 | 0.9655 | 0.9128 | 0.9496 | | 0.0486 | 1.65 | 280 | 0.1474 | 0.8464 | 0.7834 | 0.8137 | 0.6859 | 0.9688 | 0.8849 | 0.9557 | | 0.0529 | 1.71 | 290 | 0.1470 | 0.8114 | 0.8136 | 0.8125 | 0.6842 | 0.9673 | 0.8978 | 0.9535 | | 0.0585 | 1.76 | 300 | 0.1625 | 0.8193 | 0.8057 | 0.8124 | 0.6841 | 0.9676 | 0.8944 | 0.9527 | | 0.0373 | 1.82 | 310 | 0.1414 | 0.8038 | 0.8240 | 0.8138 | 0.6860 | 0.9672 | 0.9024 | 0.9555 | | 0.0474 | 1.88 | 320 | 0.1600 | 0.8117 | 0.8073 | 0.8095 | 0.6800 | 0.9670 | 0.8947 | 0.9541 | | 0.0475 | 1.94 | 330 | 0.1825 | 0.7709 | 0.8418 | 0.8048 | 0.6733 | 0.9645 | 0.9090 | 0.9498 | | 0.0596 | 2.0 | 340 | 0.1688 | 0.8185 | 0.8078 | 0.8131 | 0.6851 | 0.9677 | 0.8954 | 0.9533 | | 0.0391 | 2.06 | 350 | 0.1775 | 0.8006 | 0.8142 | 0.8073 | 0.6769 | 0.9662 | 0.8974 | 0.9514 | | 0.0285 | 2.12 | 360 | 0.1653 | 0.8047 | 0.8165 | 0.8105 | 0.6814 | 0.9668 | 0.8988 | 0.9536 | | 0.0288 | 2.18 | 370 | 0.1855 | 0.7697 | 0.8376 | 0.8022 | 0.6698 | 0.9641 | 0.9069 | 0.9509 | | 0.0244 | 2.24 | 380 | 0.1813 | 0.8053 | 0.8171 | 0.8112 | 0.6823 | 0.9669 | 0.8991 | 0.9530 | | 0.0324 | 2.29 | 390 | 0.1663 | 0.8029 | 0.8252 | 0.8139 | 0.6862 | 0.9672 | 0.9030 | 0.9538 | | 0.0306 | 2.35 | 400 | 0.1692 | 0.7949 | 0.8291 | 0.8117 | 0.6830 | 0.9665 | 0.9044 | 0.9529 | | 0.0279 | 2.41 | 410 | 0.1812 | 0.8171 | 0.8056 | 0.8113 | 0.6825 | 0.9674 | 0.8942 | 0.9526 | | 0.0287 | 2.47 | 420 | 0.1768 | 0.8196 | 0.8106 | 0.8151 | 0.6879 | 0.9680 | 0.8968 | 0.9540 | | 0.0287 | 2.53 | 430 | 0.1849 | 0.7927 | 0.8271 | 0.8095 | 0.6800 | 0.9662 | 0.9032 | 0.9532 | | 0.0328 | 2.59 | 440 | 0.1765 | 0.8031 | 0.8197 | 0.8113 | 0.6825 | 0.9668 | 0.9003 | 0.9536 | | 0.0226 | 2.65 | 450 | 0.1928 | 0.7879 | 0.8403 | 0.8133 | 0.6853 | 0.9664 | 0.9094 | 0.9515 | | 0.0308 | 2.71 | 460 | 0.1905 | 0.7858 | 0.8354 | 0.8098 | 0.6804 | 0.9659 | 0.9069 | 0.9515 | | 0.0277 | 2.76 | 470 | 0.1890 | 0.8083 | 0.8168 | 0.8125 | 0.6842 | 0.9672 | 0.8992 | 0.9537 | | 0.0229 | 2.82 | 480 | 0.1802 | 0.8125 | 0.8177 | 0.8151 | 0.6879 | 0.9677 | 0.8999 | 0.9540 | | 0.0218 | 2.88 | 490 | 0.1895 | 0.7805 | 0.8370 | 0.8078 | 0.6775 | 0.9654 | 0.9073 | 0.9506 | | 0.0283 | 2.94 | 500 | 0.1926 | 0.7854 | 0.8337 | 0.8088 | 0.6790 | 0.9657 | 0.9060 | 0.9516 | | 0.0231 | 3.0 | 510 | 0.2023 | 0.7766 | 0.8432 | 0.8085 | 0.6786 | 0.9653 | 0.9101 | 0.9509 | | 0.0172 | 3.06 | 520 | 0.2051 | 0.7811 | 0.8305 | 0.8051 | 0.6737 | 0.9650 | 0.9042 | 0.9509 | | 0.0206 | 3.12 | 530 | 0.1918 | 0.7894 | 0.8339 | 0.8110 | 0.6822 | 0.9662 | 0.9064 | 0.9523 | | 0.0243 | 3.18 | 540 | 0.1982 | 0.7992 | 0.8198 | 0.8094 | 0.6798 | 0.9664 | 0.9001 | 0.9526 | | 0.0193 | 3.24 | 550 | 0.2036 | 0.8024 | 0.8165 | 0.8094 | 0.6798 | 0.9666 | 0.8987 | 0.9517 | | 0.0212 | 3.29 | 560 | 0.1967 | 0.8093 | 0.8175 | 0.8134 | 0.6854 | 0.9674 | 0.8996 | 0.9535 | | 0.0188 | 3.35 | 570 | 0.1944 | 0.8056 | 0.8188 | 0.8122 | 0.6837 | 0.9671 | 0.9000 | 0.9527 | | 0.0176 | 3.41 | 580 | 0.1975 | 0.8014 | 0.8239 | 0.8125 | 0.6842 | 0.9669 | 0.9022 | 0.9528 | | 0.0197 | 3.47 | 590 | 0.2118 | 0.8058 | 0.8186 | 0.8122 | 0.6837 | 0.9671 | 0.8999 | 0.9529 | | 0.0142 | 3.53 | 600 | 0.2000 | 0.8107 | 0.8187 | 0.8147 | 0.6873 | 0.9676 | 0.9003 | 0.9540 | | 0.0145 | 3.59 | 610 | 0.2095 | 0.7950 | 0.8278 | 0.8111 | 0.6822 | 0.9665 | 0.9037 | 0.9522 | | 0.023 | 3.65 | 620 | 0.2107 | 0.7881 | 0.8268 | 0.8070 | 0.6764 | 0.9656 | 0.9028 | 0.9511 | | 0.0156 | 3.71 | 630 | 0.2191 | 0.7814 | 0.8366 | 0.8081 | 0.6780 | 0.9654 | 0.9072 | 0.9507 | | 0.0175 | 3.76 | 640 | 0.2090 | 0.8081 | 0.8202 | 0.8141 | 0.6865 | 0.9674 | 0.9008 | 0.9528 | | 0.02 | 3.82 | 650 | 0.2160 | 0.8068 | 0.8224 | 0.8145 | 0.6871 | 0.9674 | 0.9018 | 0.9524 | | 0.0148 | 3.88 | 660 | 0.2049 | 0.7972 | 0.8306 | 0.8136 | 0.6857 | 0.9669 | 0.9052 | 0.9526 | | 0.0184 | 3.94 | 670 | 0.2122 | 0.7908 | 0.8342 | 0.8119 | 0.6834 | 0.9664 | 0.9066 | 0.9513 | | 0.0178 | 4.0 | 680 | 0.2090 | 0.7925 | 0.8271 | 0.8094 | 0.6799 | 0.9661 | 0.9032 | 0.9519 | | 0.0159 | 4.06 | 690 | 0.2137 | 0.7963 | 0.8293 | 0.8125 | 0.6842 | 0.9667 | 0.9045 | 0.9528 | | 0.0172 | 4.12 | 700 | 0.2150 | 0.7917 | 0.8317 | 0.8112 | 0.6824 | 0.9663 | 0.9054 | 0.9520 | | 0.0163 | 4.18 | 710 | 0.2174 | 0.8010 | 0.8250 | 0.8128 | 0.6846 | 0.9670 | 0.9027 | 0.9524 | | 0.0117 | 4.24 | 720 | 0.2175 | 0.8036 | 0.8229 | 0.8131 | 0.6851 | 0.9671 | 0.9019 | 0.9530 | | 0.0149 | 4.29 | 730 | 0.2225 | 0.7990 | 0.8242 | 0.8114 | 0.6826 | 0.9667 | 0.9022 | 0.9515 | | 0.014 | 4.35 | 740 | 0.2157 | 0.7916 | 0.8273 | 0.8090 | 0.6793 | 0.9660 | 0.9033 | 0.9520 | | 0.0139 | 4.41 | 750 | 0.2217 | 0.7952 | 0.8231 | 0.8089 | 0.6792 | 0.9662 | 0.9015 | 0.9512 | | 0.0143 | 4.47 | 760 | 0.2201 | 0.7914 | 0.8317 | 0.8111 | 0.6822 | 0.9663 | 0.9054 | 0.9518 | | 0.0147 | 4.53 | 770 | 0.2218 | 0.7945 | 0.8264 | 0.8101 | 0.6809 | 0.9663 | 0.9030 | 0.9517 | | 0.0124 | 4.59 | 780 | 0.2300 | 0.7915 | 0.8337 | 0.8120 | 0.6835 | 0.9664 | 0.9064 | 0.9515 | | 0.0157 | 4.65 | 790 | 0.2244 | 0.8038 | 0.8191 | 0.8114 | 0.6826 | 0.9669 | 0.9000 | 0.9520 | | 0.0126 | 4.71 | 800 | 0.2324 | 0.7983 | 0.8256 | 0.8117 | 0.6831 | 0.9667 | 0.9029 | 0.9514 | | 0.0137 | 4.76 | 810 | 0.2263 | 0.8000 | 0.8226 | 0.8112 | 0.6823 | 0.9667 | 0.9015 | 0.9520 | | 0.0133 | 4.82 | 820 | 0.2295 | 0.7932 | 0.8299 | 0.8111 | 0.6823 | 0.9664 | 0.9046 | 0.9512 | | 0.0155 | 4.88 | 830 | 0.2266 | 0.8019 | 0.8226 | 0.8121 | 0.6837 | 0.9669 | 0.9016 | 0.9520 | | 0.0135 | 4.94 | 840 | 0.2286 | 0.7932 | 0.8318 | 0.8121 | 0.6836 | 0.9665 | 0.9056 | 0.9518 | | 0.0073 | 5.0 | 850 | 0.2283 | 0.8019 | 0.8235 | 0.8125 | 0.6843 | 0.9670 | 0.9021 | 0.9520 | | 0.0147 | 5.06 | 860 | 0.2292 | 0.7965 | 0.8290 | 0.8124 | 0.6841 | 0.9667 | 0.9044 | 0.9518 | | 0.0139 | 5.12 | 870 | 0.2293 | 0.8013 | 0.8251 | 0.8130 | 0.6850 | 0.9670 | 0.9028 | 0.9520 | | 0.0162 | 5.18 | 880 | 0.2277 | 0.7977 | 0.8261 | 0.8116 | 0.6830 | 0.9667 | 0.9031 | 0.9518 | | 0.0111 | 5.24 | 890 | 0.2285 | 0.7941 | 0.8288 | 0.8111 | 0.6822 | 0.9664 | 0.9042 | 0.9517 | | 0.0128 | 5.29 | 900 | 0.2273 | 0.7987 | 0.8240 | 0.8112 | 0.6823 | 0.9666 | 0.9021 | 0.9521 | | 0.0129 | 5.35 | 910 | 0.2282 | 0.7989 | 0.8244 | 0.8114 | 0.6827 | 0.9667 | 0.9023 | 0.9520 | | 0.0115 | 5.41 | 920 | 0.2285 | 0.7991 | 0.8246 | 0.8117 | 0.6830 | 0.9667 | 0.9024 | 0.9520 | | 0.011 | 5.47 | 930 | 0.2283 | 0.7999 | 0.8242 | 0.8119 | 0.6833 | 0.9668 | 0.9023 | 0.9522 | | 0.013 | 5.53 | 940 | 0.2281 | 0.7995 | 0.8239 | 0.8115 | 0.6828 | 0.9667 | 0.9021 | 0.9522 | | 0.0132 | 5.59 | 950 | 0.2281 | 0.7993 | 0.8239 | 0.8114 | 0.6826 | 0.9667 | 0.9021 | 0.9522 | | 0.0104 | 5.65 | 960 | 0.2281 | 0.7992 | 0.8244 | 0.8116 | 0.6829 | 0.9667 | 0.9023 | 0.9522 | | 0.0143 | 5.71 | 970 | 0.2279 | 0.7988 | 0.8245 | 0.8114 | 0.6827 | 0.9667 | 0.9024 | 0.9521 | | 0.0133 | 5.76 | 980 | 0.2280 | 0.7988 | 0.8241 | 0.8113 | 0.6825 | 0.9667 | 0.9022 | 0.9521 | | 0.0079 | 5.82 | 990 | 0.2282 | 0.7986 | 0.8244 | 0.8113 | 0.6825 | 0.9666 | 0.9023 | 0.9521 | | 0.0108 | 5.88 | 1000 | 0.2282 | 0.7986 | 0.8244 | 0.8113 | 0.6825 | 0.9666 | 0.9023 | 0.9521 | ### Framework versions - Transformers 4.34.1 - Pytorch 2.0.1 - Datasets 2.14.6 - Tokenizers 0.14.1