trainer: training complete at 2024-02-06 19:19:02.015127.

README.md

@@ -16,13 +16,13 @@ model-index:
     dataset:
       name: fancy_dataset
       type: fancy_dataset
-      config: simple
+      config: sep_tok
       split: test
-      args: simple
+      args: sep_tok
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.8226736894908453
+      value: 0.8782096113497851
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the fancy_dataset dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4587
-- Claim: {'precision': 0.5772692208794035, 'recall': 0.5279868297271872, 'f1-score': 0.5515292961552635, 'support': 4252.0}
-- Majorclaim: {'precision': 0.6656682890303257, 'recall': 0.8148487626031164, 'f1-score': 0.7327426334226252, 'support': 2182.0}
-- O: {'precision': 0.9301160937855679, 'recall': 0.8810781671159029, 'f1-score': 0.9049332816566081, 'support': 9275.0}
-- Premise: {'precision': 0.8568813181564913, 'recall': 0.8823770491803279, 'f1-score': 0.8694423131284578, 'support': 12200.0}
-- Accuracy: 0.8227
-- Macro avg: {'precision': 0.757483730462947, 'recall': 0.7765727021566337, 'f1-score': 0.7646618810907386, 'support': 27909.0}
-- Weighted avg: {'precision': 0.8236703495364839, 'recall': 0.8226736894908453, 'f1-score': 0.8221147085496641, 'support': 27909.0}
+- Loss: 0.2733
+- Claim: {'precision': 0.5897947548460661, 'recall': 0.4865945437441204, 'f1-score': 0.5332474226804124, 'support': 4252.0}
+- Majorclaim: {'precision': 0.7717437420449724, 'recall': 0.8336388634280477, 'f1-score': 0.8014981273408239, 'support': 2182.0}
+- O: {'precision': 0.9999121265377856, 'recall': 0.9987711752830686, 'f1-score': 0.9993413252535898, 'support': 11393.0}
+- Premise: {'precision': 0.8686434047879831, 'recall': 0.9100819672131147, 'f1-score': 0.8888799935953887, 'support': 12200.0}
+- Accuracy: 0.8782
+- Macro avg: {'precision': 0.8075235070542018, 'recall': 0.8072716374170879, 'f1-score': 0.8057417172175537, 'support': 30027.0}
+- Weighted avg: {'precision': 0.8719219548674857, 'recall': 0.8782096113497851, 'f1-score': 0.874082279134535, 'support': 30027.0}
 
 ## Model description
 
@@ -68,11 +68,11 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                                | Majorclaim                                                                                                         | O                                                                                                                  | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
-|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 41   | 0.5844          | {'precision': 0.4909433962264151, 'recall': 0.30597365945437444, 'f1-score': 0.37699217618081715, 'support': 4252.0} | {'precision': 0.5969423210562891, 'recall': 0.3936755270394134, 'f1-score': 0.4744545705606187, 'support': 2182.0} | {'precision': 0.825323567773653, 'recall': 0.886900269541779, 'f1-score': 0.8550046772684753, 'support': 9275.0}   | {'precision': 0.8012704829278856, 'recall': 0.9098360655737705, 'f1-score': 0.8521091620926572, 'support': 12200.0} | 0.7699   | {'precision': 0.6786199419960607, 'recall': 0.6240963804023343, 'f1-score': 0.639640146525642, 'support': 27909.0}  | {'precision': 0.7460100844931877, 'recall': 0.7698591852090724, 'f1-score': 0.7511602266394221, 'support': 27909.0} |
-| No log        | 2.0   | 82   | 0.4763          | {'precision': 0.5736620565243535, 'recall': 0.4487300094073377, 'f1-score': 0.5035629453681709, 'support': 4252.0}   | {'precision': 0.7095454545454546, 'recall': 0.7153987167736022, 'f1-score': 0.7124600638977636, 'support': 2182.0} | {'precision': 0.8935006435006435, 'recall': 0.8982210242587602, 'f1-score': 0.8958546158395613, 'support': 9275.0} | {'precision': 0.8362814916915537, 'recall': 0.8951639344262295, 'f1-score': 0.8647214854111407, 'support': 12200.0} | 0.8141   | {'precision': 0.7532474115655013, 'recall': 0.7393784212164823, 'f1-score': 0.7441497776291592, 'support': 27909.0} | {'precision': 0.8053779036606528, 'recall': 0.8141101436812498, 'f1-score': 0.80814042735527, 'support': 27909.0}   |
-| No log        | 3.0   | 123  | 0.4587          | {'precision': 0.5772692208794035, 'recall': 0.5279868297271872, 'f1-score': 0.5515292961552635, 'support': 4252.0}   | {'precision': 0.6656682890303257, 'recall': 0.8148487626031164, 'f1-score': 0.7327426334226252, 'support': 2182.0} | {'precision': 0.9301160937855679, 'recall': 0.8810781671159029, 'f1-score': 0.9049332816566081, 'support': 9275.0} | {'precision': 0.8568813181564913, 'recall': 0.8823770491803279, 'f1-score': 0.8694423131284578, 'support': 12200.0} | 0.8227   | {'precision': 0.757483730462947, 'recall': 0.7765727021566337, 'f1-score': 0.7646618810907386, 'support': 27909.0}  | {'precision': 0.8236703495364839, 'recall': 0.8226736894908453, 'f1-score': 0.8221147085496641, 'support': 27909.0} |
+| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                              | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.3640          | {'precision': 0.4976234003656307, 'recall': 0.3200846660395108, 'f1-score': 0.3895806497781594, 'support': 4252.0} | {'precision': 0.655590480466996, 'recall': 0.6691109074243813, 'f1-score': 0.6622816965298253, 'support': 2182.0}  | {'precision': 0.9937212592854616, 'recall': 0.986307381725621, 'f1-score': 0.9900004405092286, 'support': 11393.0}  | {'precision': 0.8257614305444501, 'recall': 0.9311475409836065, 'f1-score': 0.8752937550564395, 'support': 12200.0} | 0.8465   | {'precision': 0.7431741426656346, 'recall': 0.7266626240432799, 'f1-score': 0.7292891354684132, 'support': 30027.0} | {'precision': 0.830657371246385, 'recall': 0.8465048123355646, 'f1-score': 0.8345573788621913, 'support': 30027.0}  |
+| No log        | 2.0   | 82   | 0.2836          | {'precision': 0.5498290180752321, 'recall': 0.5293979303857008, 'f1-score': 0.5394200814761563, 'support': 4252.0} | {'precision': 0.7881438289601554, 'recall': 0.7433547204399633, 'f1-score': 0.7650943396226415, 'support': 2182.0} | {'precision': 0.9999119563303398, 'recall': 0.9968401650136048, 'f1-score': 0.9983736978594347, 'support': 11393.0} | {'precision': 0.8714548214428377, 'recall': 0.8940983606557377, 'f1-score': 0.8826313873042844, 'support': 12200.0} | 0.8705   | {'precision': 0.8023349062021413, 'recall': 0.7909227941237517, 'f1-score': 0.7963798765656291, 'support': 30027.0} | {'precision': 0.8685965484304501, 'recall': 0.8704832317580844, 'f1-score': 0.8694050188269901, 'support': 30027.0} |
+| No log        | 3.0   | 123  | 0.2733          | {'precision': 0.5897947548460661, 'recall': 0.4865945437441204, 'f1-score': 0.5332474226804124, 'support': 4252.0} | {'precision': 0.7717437420449724, 'recall': 0.8336388634280477, 'f1-score': 0.8014981273408239, 'support': 2182.0} | {'precision': 0.9999121265377856, 'recall': 0.9987711752830686, 'f1-score': 0.9993413252535898, 'support': 11393.0} | {'precision': 0.8686434047879831, 'recall': 0.9100819672131147, 'f1-score': 0.8888799935953887, 'support': 12200.0} | 0.8782   | {'precision': 0.8075235070542018, 'recall': 0.8072716374170879, 'f1-score': 0.8057417172175537, 'support': 30027.0} | {'precision': 0.8719219548674857, 'recall': 0.8782096113497851, 'f1-score': 0.874082279134535, 'support': 30027.0}  |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0c66b0115b428110d25620ff4545f7258009c416d5f3270ec63ca3a4093ca074
+oid sha256:5626d1e820f82cf13e292cc22e09fc71b28e68cb756bfb6d84b5bbdd7326dff5
 size 592324828