Initial Commit

README.md

@@ -18,9 +18,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 5.0325
-- Accuracy: 0.5177
-- F1: 0.5177
+- Loss: 4.6543
+- Accuracy: 0.5131
+- F1: 0.5145
 
 ## Model description
 
@@ -40,8 +40,8 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 5e-05
-- train_batch_size: 32
-- eval_batch_size: 64
+- train_batch_size: 64
+- eval_batch_size: 128
 - seed: 66
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
@@ -51,33 +51,19 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
-| No log        | 1.09  | 250  | 1.2368          | 0.5139   | 0.5065 |
-| 0.7153        | 2.17  | 500  | 1.6085          | 0.5162   | 0.5159 |
-| 0.7153        | 3.26  | 750  | 1.7398          | 0.5123   | 0.5119 |
-| 0.3337        | 4.35  | 1000 | 2.1884          | 0.4931   | 0.4900 |
-| 0.3337        | 5.43  | 1250 | 2.5671          | 0.5324   | 0.5325 |
-| 0.1697        | 6.52  | 1500 | 2.7392          | 0.5216   | 0.5104 |
-| 0.1697        | 7.61  | 1750 | 2.3740          | 0.5054   | 0.5054 |
-| 0.1105        | 8.7   | 2000 | 3.1334          | 0.5108   | 0.5071 |
-| 0.1105        | 9.78  | 2250 | 3.1748          | 0.5262   | 0.5275 |
-| 0.0727        | 10.87 | 2500 | 3.8017          | 0.5162   | 0.5104 |
-| 0.0727        | 11.96 | 2750 | 3.9002          | 0.4969   | 0.4934 |
-| 0.0494        | 13.04 | 3000 | 3.3222          | 0.5363   | 0.5383 |
-| 0.0494        | 14.13 | 3250 | 3.8319          | 0.5231   | 0.5245 |
-| 0.032         | 15.22 | 3500 | 4.0484          | 0.5239   | 0.5246 |
-| 0.032         | 16.3  | 3750 | 4.2498          | 0.5170   | 0.5181 |
-| 0.0204        | 17.39 | 4000 | 4.2426          | 0.5108   | 0.5121 |
-| 0.0204        | 18.48 | 4250 | 4.6214          | 0.5093   | 0.5071 |
-| 0.0111        | 19.57 | 4500 | 4.6396          | 0.5185   | 0.5160 |
-| 0.0111        | 20.65 | 4750 | 4.5908          | 0.5270   | 0.5266 |
-| 0.007         | 21.74 | 5000 | 4.9185          | 0.5201   | 0.5188 |
-| 0.007         | 22.83 | 5250 | 5.1551          | 0.5154   | 0.5140 |
-| 0.0069        | 23.91 | 5500 | 4.9583          | 0.5239   | 0.5248 |
-| 0.0069        | 25.0  | 5750 | 4.9928          | 0.5116   | 0.5093 |
-| 0.006         | 26.09 | 6000 | 4.7920          | 0.5255   | 0.5239 |
-| 0.006         | 27.17 | 6250 | 4.9030          | 0.5185   | 0.5195 |
-| 0.0031        | 28.26 | 6500 | 4.9916          | 0.5170   | 0.5176 |
-| 0.0031        | 29.35 | 6750 | 5.0325          | 0.5177   | 0.5177 |
+| No log        | 2.17  | 250  | 1.4134          | 0.5069   | 0.5083 |
+| 0.5524        | 4.35  | 500  | 1.9853          | 0.5208   | 0.5230 |
+| 0.5524        | 6.52  | 750  | 2.5990          | 0.4853   | 0.4797 |
+| 0.1315        | 8.7   | 1000 | 2.8603          | 0.4961   | 0.4954 |
+| 0.1315        | 10.87 | 1250 | 3.1408          | 0.5093   | 0.5099 |
+| 0.0497        | 13.04 | 1500 | 3.3859          | 0.5177   | 0.5190 |
+| 0.0497        | 15.22 | 1750 | 3.9204          | 0.5039   | 0.5044 |
+| 0.0219        | 17.39 | 2000 | 4.0747          | 0.5139   | 0.5160 |
+| 0.0219        | 19.57 | 2250 | 4.3170          | 0.5139   | 0.5156 |
+| 0.0133        | 21.74 | 2500 | 4.5924          | 0.5023   | 0.5020 |
+| 0.0133        | 23.91 | 2750 | 4.6042          | 0.5100   | 0.5114 |
+| 0.0046        | 26.09 | 3000 | 4.5407          | 0.5147   | 0.5163 |
+| 0.0046        | 28.26 | 3250 | 4.6543          | 0.5131   | 0.5145 |
 
 
 ### Framework versions

eval_results_cardiff.json

@@ -0,0 +1 @@
+{"arabic": {"f1": 0.602933121281396, "accuracy": 0.6011494252873564, "confusion_matrix": [[149, 105, 36], [46, 204, 40], [18, 102, 170]]}, "english": {"f1": 0.7045149108105302, "accuracy": 0.7103448275862069, "confusion_matrix": [[250, 34, 6], [91, 151, 48], [21, 52, 217]]}, "french": {"f1": 0.5256412382793743, "accuracy": 0.5494252873563218, "confusion_matrix": [[152, 125, 13], [21, 257, 12], [21, 200, 69]]}, "german": {"f1": 0.7476717511679669, "accuracy": 0.7494252873563219, "confusion_matrix": [[231, 32, 27], [55, 188, 47], [25, 32, 233]]}, "hindi": {"f1": 0.5333296333296333, "accuracy": 0.532183908045977, "confusion_matrix": [[147, 93, 50], [61, 150, 79], [41, 83, 166]]}, "italian": {"f1": 0.7290103738322573, "accuracy": 0.728735632183908, "confusion_matrix": [[193, 63, 34], [23, 239, 28], [21, 67, 202]]}, "portuguese": {"f1": 0.664227795592618, "accuracy": 0.667816091954023, "confusion_matrix": [[186, 54, 50], [50, 161, 79], [24, 32, 234]]}, "spanish": {"f1": 0.6453352385900618, "accuracy": 0.6494252873563219, "confusion_matrix": [[223, 47, 20], [87, 142, 61], [26, 64, 200]]}, "all": {"f1": 0.6441442370542694, "accuracy": 0.6422413793103449, "confusion_matrix": [[1518, 558, 244], [439, 1439, 442], [194, 613, 1513]]}}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:535c719708381fcc9ad5c9e0358e31844f093f99ff7e15784c0933ac81017c35
+oid sha256:4a03fb5668e091e8dd92052136aae174bab9ff4915d964dc5ea826bec8423d13
 size 1115316658

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d1445cb95f9b9d5bf1e039b72b35923595c0c56349201043f03c70d2a454243b
-size 4600
+oid sha256:52453acfb8e342e114302b737611fd2cec5ad6385d83476c890b56eb1f71e974
+size 4536