Hugging Face's logo

haryoaw
/
scenario-NON-KD-SCR-COPY-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual

Text Classification
Transformers
PyTorch
Safetensors
deberta-v2
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
haryoaw commited on
Commit
eb41bcb
•
1 Parent(s): 324c284

Initial Commit

Browse files
Files changed (5) hide show
  1. README.md +56 -55
  2. config.json +1 -1
  3. eval_results_cardiff.json +1 -1
  4. model.safetensors +1 -1
  5. training_args.bin +2 -2
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  license: mit
3
  base_model: microsoft/mdeberta-v3-base
4
  tags:
@@ -23,10 +24,10 @@ model-index:
23
  metrics:
24
  - name: Accuracy
25
  type: accuracy
26
- value: 0.4957561728395062
27
  - name: F1
28
  type: f1
29
- value: 0.4963309489168229
30
  ---
31
 
32
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -36,9 +37,9 @@ should probably proofread and complete it, then remove this comment. -->
36
 
37
  This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the tweet_sentiment_multilingual dataset.
38
  It achieves the following results on the evaluation set:
39
- - Loss: 6.7830
40
- - Accuracy: 0.4958
41
- - F1: 0.4963
42
 
43
  ## Model description
44
 
@@ -67,59 +68,59 @@ The following hyperparameters were used during training:
67
 
68
  ### Training results
69
 
70
- | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
71
- |:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
72
- | 1.0563 | 1.09 | 500 | 1.0531 | 0.4950 | 0.4889 |
73
- | 0.7646 | 2.17 | 1000 | 1.2908 | 0.5131 | 0.5122 |
74
- | 0.4473 | 3.26 | 1500 | 1.6933 | 0.5158 | 0.5165 |
75
- | 0.2465 | 4.35 | 2000 | 2.2069 | 0.5135 | 0.5126 |
76
- | 0.1511 | 5.43 | 2500 | 2.5482 | 0.5081 | 0.5082 |
77
- | 0.121 | 6.52 | 3000 | 2.8075 | 0.5123 | 0.5107 |
78
- | 0.0834 | 7.61 | 3500 | 3.1416 | 0.5085 | 0.5082 |
79
- | 0.061 | 8.7 | 4000 | 2.8363 | 0.5093 | 0.5083 |
80
- | 0.0496 | 9.78 | 4500 | 3.3908 | 0.5174 | 0.5162 |
81
- | 0.0394 | 10.87 | 5000 | 3.6362 | 0.5123 | 0.5126 |
82
- | 0.0305 | 11.96 | 5500 | 4.0351 | 0.5035 | 0.5047 |
83
- | 0.0283 | 13.04 | 6000 | 4.0528 | 0.5031 | 0.5042 |
84
- | 0.0184 | 14.13 | 6500 | 4.2723 | 0.5039 | 0.5045 |
85
- | 0.0217 | 15.22 | 7000 | 4.2612 | 0.4981 | 0.4977 |
86
- | 0.0193 | 16.3 | 7500 | 4.3257 | 0.4907 | 0.4915 |
87
- | 0.0196 | 17.39 | 8000 | 4.6089 | 0.4904 | 0.4906 |
88
- | 0.0154 | 18.48 | 8500 | 4.6472 | 0.4927 | 0.4935 |
89
- | 0.014 | 19.57 | 9000 | 4.4510 | 0.4981 | 0.4982 |
90
- | 0.0177 | 20.65 | 9500 | 4.2732 | 0.4907 | 0.4911 |
91
- | 0.0114 | 21.74 | 10000 | 4.5261 | 0.4931 | 0.4921 |
92
- | 0.0099 | 22.83 | 10500 | 4.9751 | 0.4888 | 0.4901 |
93
- | 0.0073 | 23.91 | 11000 | 4.4316 | 0.4927 | 0.4923 |
94
- | 0.0081 | 25.0 | 11500 | 4.8393 | 0.4942 | 0.4940 |
95
- | 0.0039 | 26.09 | 12000 | 5.2291 | 0.4988 | 0.4958 |
96
- | 0.0052 | 27.17 | 12500 | 5.1648 | 0.4931 | 0.4942 |
97
- | 0.0065 | 28.26 | 13000 | 5.1350 | 0.4919 | 0.4924 |
98
- | 0.0042 | 29.35 | 13500 | 5.2707 | 0.4988 | 0.4971 |
99
- | 0.0033 | 30.43 | 14000 | 5.2902 | 0.4896 | 0.4911 |
100
- | 0.0041 | 31.52 | 14500 | 5.3182 | 0.4958 | 0.4971 |
101
- | 0.002 | 32.61 | 15000 | 5.4473 | 0.4961 | 0.4968 |
102
- | 0.001 | 33.7 | 15500 | 5.7540 | 0.4942 | 0.4952 |
103
- | 0.0016 | 34.78 | 16000 | 5.8709 | 0.4958 | 0.4929 |
104
- | 0.001 | 35.87 | 16500 | 6.1489 | 0.4938 | 0.4936 |
105
- | 0.0012 | 36.96 | 17000 | 6.4545 | 0.4942 | 0.4942 |
106
- | 0.0011 | 38.04 | 17500 | 6.4864 | 0.4946 | 0.4936 |
107
- | 0.0024 | 39.13 | 18000 | 6.2903 | 0.5012 | 0.4998 |
108
- | 0.001 | 40.22 | 18500 | 6.2566 | 0.4954 | 0.4950 |
109
- | 0.0002 | 41.3 | 19000 | 6.3660 | 0.4954 | 0.4955 |
110
- | 0.001 | 42.39 | 19500 | 6.4778 | 0.4954 | 0.4923 |
111
- | 0.0001 | 43.48 | 20000 | 6.5401 | 0.4985 | 0.4981 |
112
- | 0.0002 | 44.57 | 20500 | 6.6695 | 0.5 | 0.4992 |
113
- | 0.0 | 45.65 | 21000 | 6.7149 | 0.5012 | 0.5004 |
114
- | 0.0001 | 46.74 | 21500 | 6.7514 | 0.5015 | 0.5011 |
115
- | 0.0008 | 47.83 | 22000 | 6.7485 | 0.4958 | 0.4964 |
116
- | 0.0001 | 48.91 | 22500 | 6.7745 | 0.4961 | 0.4968 |
117
- | 0.0001 | 50.0 | 23000 | 6.7830 | 0.4958 | 0.4963 |
118
 
119
 
120
  ### Framework versions
121
 
122
- - Transformers 4.33.3
123
  - Pytorch 2.1.1+cu121
124
  - Datasets 2.14.5
125
- - Tokenizers 0.13.3
 
1
  ---
2
+ library_name: transformers
3
  license: mit
4
  base_model: microsoft/mdeberta-v3-base
5
  tags:
 
24
  metrics:
25
  - name: Accuracy
26
  type: accuracy
27
+ value: 0.4903549382716049
28
  - name: F1
29
  type: f1
30
+ value: 0.490123758683559
31
  ---
32
 
33
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
37
 
38
  This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the tweet_sentiment_multilingual dataset.
39
  It achieves the following results on the evaluation set:
40
+ - Loss: 6.8615
41
+ - Accuracy: 0.4904
42
+ - F1: 0.4901
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
72
+ |:-------------:|:-------:|:-----:|:---------------:|:--------:|:------:|
73
+ | 1.0475 | 1.0870 | 500 | 1.0371 | 0.4985 | 0.4949 |
74
+ | 0.7462 | 2.1739 | 1000 | 1.2759 | 0.5123 | 0.5122 |
75
+ | 0.421 | 3.2609 | 1500 | 1.6791 | 0.5139 | 0.5126 |
76
+ | 0.2321 | 4.3478 | 2000 | 2.1227 | 0.4946 | 0.4940 |
77
+ | 0.1534 | 5.4348 | 2500 | 2.4070 | 0.4958 | 0.4966 |
78
+ | 0.0987 | 6.5217 | 3000 | 2.8761 | 0.4904 | 0.4900 |
79
+ | 0.0734 | 7.6087 | 3500 | 2.8613 | 0.4911 | 0.4881 |
80
+ | 0.0697 | 8.6957 | 4000 | 3.5593 | 0.4969 | 0.4932 |
81
+ | 0.0586 | 9.7826 | 4500 | 3.4005 | 0.4900 | 0.4883 |
82
+ | 0.0462 | 10.8696 | 5000 | 3.6698 | 0.4861 | 0.4866 |
83
+ | 0.0321 | 11.9565 | 5500 | 4.1118 | 0.4877 | 0.4883 |
84
+ | 0.0267 | 13.0435 | 6000 | 4.1028 | 0.4965 | 0.4959 |
85
+ | 0.0257 | 14.1304 | 6500 | 4.3167 | 0.4842 | 0.4815 |
86
+ | 0.0185 | 15.2174 | 7000 | 4.3273 | 0.4923 | 0.4876 |
87
+ | 0.0178 | 16.3043 | 7500 | 4.7543 | 0.4958 | 0.4959 |
88
+ | 0.0149 | 17.3913 | 8000 | 4.3035 | 0.4927 | 0.4929 |
89
+ | 0.0125 | 18.4783 | 8500 | 4.5842 | 0.4904 | 0.4884 |
90
+ | 0.0116 | 19.5652 | 9000 | 5.3172 | 0.4853 | 0.4833 |
91
+ | 0.0114 | 20.6522 | 9500 | 4.8280 | 0.4857 | 0.4825 |
92
+ | 0.0036 | 21.7391 | 10000 | 5.6275 | 0.4850 | 0.4820 |
93
+ | 0.0094 | 22.8261 | 10500 | 5.1559 | 0.4842 | 0.4815 |
94
+ | 0.0054 | 23.9130 | 11000 | 5.3889 | 0.4846 | 0.4826 |
95
+ | 0.0085 | 25.0 | 11500 | 4.8587 | 0.4888 | 0.4861 |
96
+ | 0.0068 | 26.0870 | 12000 | 5.3553 | 0.4896 | 0.4881 |
97
+ | 0.0054 | 27.1739 | 12500 | 5.3446 | 0.4853 | 0.4845 |
98
+ | 0.0042 | 28.2609 | 13000 | 5.3437 | 0.4838 | 0.4832 |
99
+ | 0.003 | 29.3478 | 13500 | 5.9054 | 0.4796 | 0.4784 |
100
+ | 0.0032 | 30.4348 | 14000 | 5.7871 | 0.4884 | 0.4881 |
101
+ | 0.0038 | 31.5217 | 14500 | 5.9122 | 0.4803 | 0.4787 |
102
+ | 0.0041 | 32.6087 | 15000 | 5.4601 | 0.4834 | 0.4786 |
103
+ | 0.0025 | 33.6957 | 15500 | 5.1979 | 0.4884 | 0.4853 |
104
+ | 0.0018 | 34.7826 | 16000 | 5.5286 | 0.4896 | 0.4869 |
105
+ | 0.0006 | 35.8696 | 16500 | 5.7718 | 0.4877 | 0.4859 |
106
+ | 0.0015 | 36.9565 | 17000 | 6.0193 | 0.4834 | 0.4832 |
107
+ | 0.0003 | 38.0435 | 17500 | 6.2210 | 0.4838 | 0.4828 |
108
+ | 0.0004 | 39.1304 | 18000 | 6.3234 | 0.4880 | 0.4879 |
109
+ | 0.0002 | 40.2174 | 18500 | 6.3829 | 0.4888 | 0.4885 |
110
+ | 0.0001 | 41.3043 | 19000 | 6.5514 | 0.4892 | 0.4889 |
111
+ | 0.0001 | 42.3913 | 19500 | 6.6261 | 0.4892 | 0.4891 |
112
+ | 0.0003 | 43.4783 | 20000 | 6.6971 | 0.4861 | 0.4849 |
113
+ | 0.0013 | 44.5652 | 20500 | 6.7077 | 0.4865 | 0.4849 |
114
+ | 0.0001 | 45.6522 | 21000 | 6.7350 | 0.4911 | 0.4903 |
115
+ | 0.0001 | 46.7391 | 21500 | 6.7889 | 0.4896 | 0.4888 |
116
+ | 0.0002 | 47.8261 | 22000 | 6.8318 | 0.4900 | 0.4902 |
117
+ | 0.0006 | 48.9130 | 22500 | 6.8526 | 0.4904 | 0.4901 |
118
+ | 0.0001 | 50.0 | 23000 | 6.8615 | 0.4904 | 0.4901 |
119
 
120
 
121
  ### Framework versions
122
 
123
+ - Transformers 4.44.2
124
  - Pytorch 2.1.1+cu121
125
  - Datasets 2.14.5
126
+ - Tokenizers 0.19.1
config.json CHANGED
@@ -39,7 +39,7 @@
39
  "relative_attention": true,
40
  "share_att_key": true,
41
  "torch_dtype": "float32",
42
- "transformers_version": "4.33.3",
43
  "type_vocab_size": 0,
44
  "vocab_size": 251000
45
  }
 
39
  "relative_attention": true,
40
  "share_att_key": true,
41
  "torch_dtype": "float32",
42
+ "transformers_version": "4.44.2",
43
  "type_vocab_size": 0,
44
  "vocab_size": 251000
45
  }
eval_results_cardiff.json CHANGED
@@ -1 +1 @@
1
- {"arabic": {"f1": 0.47089949037256823, "accuracy": 0.4689655172413793, "confusion_matrix": [[133, 120, 37], [97, 153, 40], [71, 97, 122]]}, "english": {"f1": 0.46593250950336423, "accuracy": 0.46781609195402296, "confusion_matrix": [[153, 84, 53], [110, 105, 75], [82, 59, 149]]}, "french": {"f1": 0.534964240110648, "accuracy": 0.535632183908046, "confusion_matrix": [[164, 71, 55], [42, 170, 78], [67, 91, 132]]}, "german": {"f1": 0.5509262447152427, "accuracy": 0.5517241379310345, "confusion_matrix": [[143, 79, 68], [60, 173, 57], [57, 69, 164]]}, "hindi": {"f1": 0.44474479625697233, "accuracy": 0.44482758620689655, "confusion_matrix": [[135, 79, 76], [77, 127, 86], [84, 81, 125]]}, "italian": {"f1": 0.4734988640051931, "accuracy": 0.49195402298850577, "confusion_matrix": [[72, 131, 87], [11, 194, 85], [32, 96, 162]]}, "portuguese": {"f1": 0.5575518820795515, "accuracy": 0.5597701149425287, "confusion_matrix": [[170, 71, 49], [82, 131, 77], [45, 59, 186]]}, "spanish": {"f1": 0.5131583962747177, "accuracy": 0.5126436781609195, "confusion_matrix": [[147, 92, 51], [81, 138, 71], [57, 72, 161]]}, "all": {"f1": 0.5044375419579221, "accuracy": 0.5041666666666667, "confusion_matrix": [[1117, 727, 476], [560, 1191, 569], [495, 624, 1201]]}}
 
1
+ {"arabic": {"f1": 0.4343944556549693, "accuracy": 0.4367816091954023, "confusion_matrix": [[158, 82, 50], [124, 118, 48], [83, 103, 104]]}, "english": {"f1": 0.4526751785985712, "accuracy": 0.45632183908045976, "confusion_matrix": [[172, 71, 47], [136, 100, 54], [82, 83, 125]]}, "french": {"f1": 0.5299219070640343, "accuracy": 0.535632183908046, "confusion_matrix": [[193, 53, 44], [72, 158, 60], [83, 92, 115]]}, "german": {"f1": 0.5447451227511411, "accuracy": 0.5448275862068965, "confusion_matrix": [[161, 69, 60], [84, 170, 36], [75, 72, 143]]}, "hindi": {"f1": 0.41369179773499637, "accuracy": 0.41724137931034483, "confusion_matrix": [[154, 72, 64], [112, 103, 75], [98, 86, 106]]}, "italian": {"f1": 0.5156494287328716, "accuracy": 0.5298850574712644, "confusion_matrix": [[88, 114, 88], [31, 198, 61], [40, 75, 175]]}, "portuguese": {"f1": 0.5755454658004749, "accuracy": 0.5770114942528736, "confusion_matrix": [[186, 58, 46], [101, 137, 52], [47, 64, 179]]}, "spanish": {"f1": 0.5305957998799439, "accuracy": 0.532183908045977, "confusion_matrix": [[182, 61, 47], [102, 130, 58], [57, 82, 151]]}, "all": {"f1": 0.5035051903366276, "accuracy": 0.503735632183908, "confusion_matrix": [[1294, 580, 446], [762, 1114, 444], [565, 657, 1098]]}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0598243c957e352f74123c5ab26c783a8030632389e1b6c8e6f1935e4082c97c
3
  size 945150428
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5ac09e5c6e40eb9576836ca7863bf8afb18e801611fc0f1f5307d247773d983
3
  size 945150428
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:534b6d504b1d4129997135ebf6daae33d3a21de1f665c5be67a95eff367d11ec
3
- size 4664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a98c3071bbd3e05b9ca810257c37a53ade2970149c1489aa66b02ea233f97695
3
+ size 5368