trainer: training complete at 2024-02-19 20:12:32.909204.

README.md

@@ -22,7 +22,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.8875012488760116
+      value: 0.8921304159589702
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.2595
-- Claim: {'precision': 0.6180851063829788, 'recall': 0.5465663217309501, 'f1-score': 0.5801298052920619, 'support': 4252.0}
-- Majorclaim: {'precision': 0.8535564853556485, 'recall': 0.8414298808432631, 'f1-score': 0.8474498038310638, 'support': 2182.0}
-- O: {'precision': 0.9998243611135506, 'recall': 0.9992978144474678, 'f1-score': 0.9995610184372257, 'support': 11393.0}
-- Premise: {'precision': 0.8723387540262393, 'recall': 0.9101639344262296, 'f1-score': 0.8908500140398733, 'support': 12200.0}
-- Accuracy: 0.8875
-- Macro avg: {'precision': 0.8359511767196043, 'recall': 0.8243644878619776, 'f1-score': 0.8294976604000562, 'support': 30027.0}
-- Weighted avg: {'precision': 0.8833413217661854, 'recall': 0.8875012488760116, 'f1-score': 0.884944092263729, 'support': 30027.0}
+- Loss: 0.2544
+- Claim: {'precision': 0.618502404396611, 'recall': 0.6352304797742239, 'f1-score': 0.6267548439494142, 'support': 4252.0}
+- Majorclaim: {'precision': 0.8701787394167451, 'recall': 0.847846012832264, 'f1-score': 0.8588672237697308, 'support': 2182.0}
+- O: {'precision': 1.0, 'recall': 0.9994733608356008, 'f1-score': 0.9997366110623354, 'support': 11393.0}
+- Premise: {'precision': 0.8932246645262205, 'recall': 0.889344262295082, 'f1-score': 0.8912802398652812, 'support': 12200.0}
+- Accuracy: 0.8921
+- Macro avg: {'precision': 0.8454764520848941, 'recall': 0.8429735289342927, 'f1-score': 0.8441597296616904, 'support': 30027.0}
+- Weighted avg: {'precision': 0.8931609265035342, 'recall': 0.8921304159589702, 'f1-score': 0.8926175780107263, 'support': 30027.0}
 
 ## Model description
 
@@ -64,16 +64,17 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 4
+- num_epochs: 5
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
 |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 41   | 0.3497          | {'precision': 0.5091044221479004, 'recall': 0.3222013170272813, 'f1-score': 0.39464208555379515, 'support': 4252.0} | {'precision': 0.6774921064501579, 'recall': 0.6883593033913841, 'f1-score': 0.682882473289384, 'support': 2182.0}  | {'precision': 0.9962913907284768, 'recall': 0.9903449486526815, 'f1-score': 0.9933092701822344, 'support': 11393.0} | {'precision': 0.8249963752356096, 'recall': 0.9327868852459016, 'f1-score': 0.875586673847811, 'support': 12200.0}  | 0.8504   | {'precision': 0.7519710736405363, 'recall': 0.7334231135793121, 'f1-score': 0.7366051257183062, 'support': 30027.0} | {'precision': 0.8345390272651644, 'recall': 0.8504013054917241, 'f1-score': 0.8381455903227649, 'support': 30027.0} |
-| No log        | 2.0   | 82   | 0.2760          | {'precision': 0.5628667225481978, 'recall': 0.6317027281279398, 'f1-score': 0.5953014184397162, 'support': 4252.0}  | {'precision': 0.8442959917780062, 'recall': 0.7529789184234648, 'f1-score': 0.7960271317829457, 'support': 2182.0} | {'precision': 0.9999119795792624, 'recall': 0.9971034845958044, 'f1-score': 0.9985057572294981, 'support': 11393.0} | {'precision': 0.8920321392701708, 'recall': 0.8736065573770492, 'f1-score': 0.8827232068908398, 'support': 12200.0} | 0.8774   | {'precision': 0.8247767082939093, 'recall': 0.8138479221310645, 'f1-score': 0.8181393785857499, 'support': 30027.0} | {'precision': 0.8828838192552426, 'recall': 0.8774436340626769, 'f1-score': 0.8796533802557692, 'support': 30027.0} |
-| No log        | 3.0   | 123  | 0.2556          | {'precision': 0.622185154295246, 'recall': 0.526340545625588, 'f1-score': 0.570263727863422, 'support': 4252.0}     | {'precision': 0.8404255319148937, 'recall': 0.8327222731439047, 'f1-score': 0.8365561694290976, 'support': 2182.0} | {'precision': 0.9998242839571253, 'recall': 0.9988589484771351, 'f1-score': 0.9993413830954994, 'support': 11393.0} | {'precision': 0.8682290858295825, 'recall': 0.9170491803278689, 'f1-score': 0.8919716176353345, 'support': 12200.0} | 0.8866   | {'precision': 0.8326660139992118, 'recall': 0.8187427368936242, 'f1-score': 0.8245332245058383, 'support': 30027.0} | {'precision': 0.8812979219018255, 'recall': 0.8866353615079762, 'f1-score': 0.8831277531997092, 'support': 30027.0} |
-| No log        | 4.0   | 164  | 0.2595          | {'precision': 0.6180851063829788, 'recall': 0.5465663217309501, 'f1-score': 0.5801298052920619, 'support': 4252.0}  | {'precision': 0.8535564853556485, 'recall': 0.8414298808432631, 'f1-score': 0.8474498038310638, 'support': 2182.0} | {'precision': 0.9998243611135506, 'recall': 0.9992978144474678, 'f1-score': 0.9995610184372257, 'support': 11393.0} | {'precision': 0.8723387540262393, 'recall': 0.9101639344262296, 'f1-score': 0.8908500140398733, 'support': 12200.0} | 0.8875   | {'precision': 0.8359511767196043, 'recall': 0.8243644878619776, 'f1-score': 0.8294976604000562, 'support': 30027.0} | {'precision': 0.8833413217661854, 'recall': 0.8875012488760116, 'f1-score': 0.884944092263729, 'support': 30027.0}  |
+| No log        | 1.0   | 41   | 0.3491          | {'precision': 0.5206252713851498, 'recall': 0.28198494825964254, 'f1-score': 0.3658276125095347, 'support': 4252.0} | {'precision': 0.626488632262721, 'recall': 0.7956003666361137, 'f1-score': 0.7009892994144962, 'support': 2182.0}  | {'precision': 0.9948045086297992, 'recall': 0.9915737733696129, 'f1-score': 0.9931865136929096, 'support': 11393.0} | {'precision': 0.8307714937118482, 'recall': 0.9259016393442623, 'f1-score': 0.8757607473737257, 'support': 12200.0} | 0.8502   | {'precision': 0.7431724764973796, 'recall': 0.7487651819024079, 'f1-score': 0.7339410432476665, 'support': 30027.0} | {'precision': 0.8342464062220922, 'recall': 0.8501681819695607, 'f1-score': 0.8354052262355796, 'support': 30027.0} |
+| No log        | 2.0   | 82   | 0.2801          | {'precision': 0.5533096401599289, 'recall': 0.5858419567262465, 'f1-score': 0.5691112634224355, 'support': 4252.0}  | {'precision': 0.7518549051937345, 'recall': 0.8359303391384051, 'f1-score': 0.7916666666666666, 'support': 2182.0} | {'precision': 0.9982446901878181, 'recall': 0.9983323093127359, 'f1-score': 0.9982884978277087, 'support': 11393.0} | {'precision': 0.8966253737718923, 'recall': 0.8602459016393442, 'f1-score': 0.8780589834762602, 'support': 12200.0} | 0.8720   | {'precision': 0.8000086523283435, 'recall': 0.820087626704183, 'f1-score': 0.8092813528482677, 'support': 30027.0}  | {'precision': 0.8760466016724828, 'recall': 0.8720151863323009, 'f1-score': 0.8736503218070512, 'support': 30027.0} |
+| No log        | 3.0   | 123  | 0.2615          | {'precision': 0.6233933161953727, 'recall': 0.45625587958607716, 'f1-score': 0.5268875611080934, 'support': 4252.0} | {'precision': 0.8267020335985853, 'recall': 0.8570119156736938, 'f1-score': 0.8415841584158417, 'support': 2182.0} | {'precision': 1.0, 'recall': 0.9987711752830686, 'f1-score': 0.9993852099069033, 'support': 11393.0}                | {'precision': 0.8530962784390538, 'recall': 0.9281967213114755, 'f1-score': 0.8890633587186937, 'support': 12200.0} | 0.8830   | {'precision': 0.825797907058253, 'recall': 0.8100589229635788, 'f1-score': 0.814230072037383, 'support': 30027.0}   | {'precision': 0.8743899428757883, 'recall': 0.8829719918739801, 'f1-score': 0.8761858066517598, 'support': 30027.0} |
+| No log        | 4.0   | 164  | 0.2551          | {'precision': 0.6193501099438065, 'recall': 0.5961900282220132, 'f1-score': 0.6075494307968844, 'support': 4252.0}  | {'precision': 0.8261056247316445, 'recall': 0.8817598533455545, 'f1-score': 0.8530259365994236, 'support': 2182.0} | {'precision': 1.0, 'recall': 0.9994733608356008, 'f1-score': 0.9997366110623354, 'support': 11393.0}                | {'precision': 0.8906531347192667, 'recall': 0.8919672131147541, 'f1-score': 0.8913096895732656, 'support': 12200.0} | 0.8901   | {'precision': 0.8340272173486795, 'recall': 0.8423476138794807, 'f1-score': 0.8379054170079773, 'support': 30027.0} | {'precision': 0.8890334493695863, 'recall': 0.890132214340427, 'f1-score': 0.88948546961186, 'support': 30027.0}    |
+| No log        | 5.0   | 205  | 0.2544          | {'precision': 0.618502404396611, 'recall': 0.6352304797742239, 'f1-score': 0.6267548439494142, 'support': 4252.0}   | {'precision': 0.8701787394167451, 'recall': 0.847846012832264, 'f1-score': 0.8588672237697308, 'support': 2182.0}  | {'precision': 1.0, 'recall': 0.9994733608356008, 'f1-score': 0.9997366110623354, 'support': 11393.0}                | {'precision': 0.8932246645262205, 'recall': 0.889344262295082, 'f1-score': 0.8912802398652812, 'support': 12200.0}  | 0.8921   | {'precision': 0.8454764520848941, 'recall': 0.8429735289342927, 'f1-score': 0.8441597296616904, 'support': 30027.0} | {'precision': 0.8931609265035342, 'recall': 0.8921304159589702, 'f1-score': 0.8926175780107263, 'support': 30027.0} |
 
 
 ### Framework versions

meta_data/README_s42_e5.md

@@ -0,0 +1,85 @@
+---
+license: apache-2.0
+base_model: allenai/longformer-base-4096
+tags:
+- generated_from_trainer
+datasets:
+- essays_su_g
+metrics:
+- accuracy
+model-index:
+- name: longformer-sep_tok
+  results:
+  - task:
+      name: Token Classification
+      type: token-classification
+    dataset:
+      name: essays_su_g
+      type: essays_su_g
+      config: sep_tok
+      split: test
+      args: sep_tok
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.8921304159589702
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# longformer-sep_tok
+
+This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2544
+- Claim: {'precision': 0.618502404396611, 'recall': 0.6352304797742239, 'f1-score': 0.6267548439494142, 'support': 4252.0}
+- Majorclaim: {'precision': 0.8701787394167451, 'recall': 0.847846012832264, 'f1-score': 0.8588672237697308, 'support': 2182.0}
+- O: {'precision': 1.0, 'recall': 0.9994733608356008, 'f1-score': 0.9997366110623354, 'support': 11393.0}
+- Premise: {'precision': 0.8932246645262205, 'recall': 0.889344262295082, 'f1-score': 0.8912802398652812, 'support': 12200.0}
+- Accuracy: 0.8921
+- Macro avg: {'precision': 0.8454764520848941, 'recall': 0.8429735289342927, 'f1-score': 0.8441597296616904, 'support': 30027.0}
+- Weighted avg: {'precision': 0.8931609265035342, 'recall': 0.8921304159589702, 'f1-score': 0.8926175780107263, 'support': 30027.0}
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 5
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.3491          | {'precision': 0.5206252713851498, 'recall': 0.28198494825964254, 'f1-score': 0.3658276125095347, 'support': 4252.0} | {'precision': 0.626488632262721, 'recall': 0.7956003666361137, 'f1-score': 0.7009892994144962, 'support': 2182.0}  | {'precision': 0.9948045086297992, 'recall': 0.9915737733696129, 'f1-score': 0.9931865136929096, 'support': 11393.0} | {'precision': 0.8307714937118482, 'recall': 0.9259016393442623, 'f1-score': 0.8757607473737257, 'support': 12200.0} | 0.8502   | {'precision': 0.7431724764973796, 'recall': 0.7487651819024079, 'f1-score': 0.7339410432476665, 'support': 30027.0} | {'precision': 0.8342464062220922, 'recall': 0.8501681819695607, 'f1-score': 0.8354052262355796, 'support': 30027.0} |
+| No log        | 2.0   | 82   | 0.2801          | {'precision': 0.5533096401599289, 'recall': 0.5858419567262465, 'f1-score': 0.5691112634224355, 'support': 4252.0}  | {'precision': 0.7518549051937345, 'recall': 0.8359303391384051, 'f1-score': 0.7916666666666666, 'support': 2182.0} | {'precision': 0.9982446901878181, 'recall': 0.9983323093127359, 'f1-score': 0.9982884978277087, 'support': 11393.0} | {'precision': 0.8966253737718923, 'recall': 0.8602459016393442, 'f1-score': 0.8780589834762602, 'support': 12200.0} | 0.8720   | {'precision': 0.8000086523283435, 'recall': 0.820087626704183, 'f1-score': 0.8092813528482677, 'support': 30027.0}  | {'precision': 0.8760466016724828, 'recall': 0.8720151863323009, 'f1-score': 0.8736503218070512, 'support': 30027.0} |
+| No log        | 3.0   | 123  | 0.2615          | {'precision': 0.6233933161953727, 'recall': 0.45625587958607716, 'f1-score': 0.5268875611080934, 'support': 4252.0} | {'precision': 0.8267020335985853, 'recall': 0.8570119156736938, 'f1-score': 0.8415841584158417, 'support': 2182.0} | {'precision': 1.0, 'recall': 0.9987711752830686, 'f1-score': 0.9993852099069033, 'support': 11393.0}                | {'precision': 0.8530962784390538, 'recall': 0.9281967213114755, 'f1-score': 0.8890633587186937, 'support': 12200.0} | 0.8830   | {'precision': 0.825797907058253, 'recall': 0.8100589229635788, 'f1-score': 0.814230072037383, 'support': 30027.0}   | {'precision': 0.8743899428757883, 'recall': 0.8829719918739801, 'f1-score': 0.8761858066517598, 'support': 30027.0} |
+| No log        | 4.0   | 164  | 0.2551          | {'precision': 0.6193501099438065, 'recall': 0.5961900282220132, 'f1-score': 0.6075494307968844, 'support': 4252.0}  | {'precision': 0.8261056247316445, 'recall': 0.8817598533455545, 'f1-score': 0.8530259365994236, 'support': 2182.0} | {'precision': 1.0, 'recall': 0.9994733608356008, 'f1-score': 0.9997366110623354, 'support': 11393.0}                | {'precision': 0.8906531347192667, 'recall': 0.8919672131147541, 'f1-score': 0.8913096895732656, 'support': 12200.0} | 0.8901   | {'precision': 0.8340272173486795, 'recall': 0.8423476138794807, 'f1-score': 0.8379054170079773, 'support': 30027.0} | {'precision': 0.8890334493695863, 'recall': 0.890132214340427, 'f1-score': 0.88948546961186, 'support': 30027.0}    |
+| No log        | 5.0   | 205  | 0.2544          | {'precision': 0.618502404396611, 'recall': 0.6352304797742239, 'f1-score': 0.6267548439494142, 'support': 4252.0}   | {'precision': 0.8701787394167451, 'recall': 0.847846012832264, 'f1-score': 0.8588672237697308, 'support': 2182.0}  | {'precision': 1.0, 'recall': 0.9994733608356008, 'f1-score': 0.9997366110623354, 'support': 11393.0}                | {'precision': 0.8932246645262205, 'recall': 0.889344262295082, 'f1-score': 0.8912802398652812, 'support': 12200.0}  | 0.8921   | {'precision': 0.8454764520848941, 'recall': 0.8429735289342927, 'f1-score': 0.8441597296616904, 'support': 30027.0} | {'precision': 0.8931609265035342, 'recall': 0.8921304159589702, 'f1-score': 0.8926175780107263, 'support': 30027.0} |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.2.0+cu121
+- Datasets 2.17.0
+- Tokenizers 0.15.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c189bbb295db7cb3313628a0dd4558b24111b337eb188511c9e73f7a763fc629
+oid sha256:2f4cea23fbeef91e3ee8309a1aec320b36b5d756ecde715a23384ebe42eb129c
 size 592324828