meta_data/README_s42_e14.md

---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: sep_tok
      split: train[80%:100%]
      args: sep_tok
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8972900185154015
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-sep_tok

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4170
- Claim: {'precision': 0.6411675893306492, 'recall': 0.6113243761996161, 'f1-score': 0.6258904446082044, 'support': 4168.0}
- Majorclaim: {'precision': 0.918905715681485, 'recall': 0.8740706319702602, 'f1-score': 0.8959276018099548, 'support': 2152.0}
- O: {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}
- Premise: {'precision': 0.8821437232236683, 'recall': 0.9039178331814793, 'f1-score': 0.8928980526918672, 'support': 12073.0}
- Accuracy: 0.8973
- Macro avg: {'precision': 0.8605321585881648, 'recall': 0.8473282103378389, 'f1-score': 0.8536679750537497, 'support': 29705.0}
- Weighted avg: {'precision': 0.8958422107843774, 'recall': 0.8972900185154015, 'f1-score': 0.8964216725962088, 'support': 29705.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 14

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.3371          | {'precision': 0.5051114023591088, 'recall': 0.4623320537428023, 'f1-score': 0.48277589878491795, 'support': 4168.0} | {'precision': 0.6749773345421578, 'recall': 0.6919144981412639, 'f1-score': 0.6833409821018817, 'support': 2152.0} | {'precision': 0.9997326917936381, 'recall': 0.9918670438472419, 'f1-score': 0.995784335478145, 'support': 11312.0}  | {'precision': 0.8627718481662788, 'recall': 0.890499461608548, 'f1-score': 0.8764164017282138, 'support': 12073.0}  | 0.8546   | {'precision': 0.760648319215296, 'recall': 0.759153264334964, 'f1-score': 0.7595794045232896, 'support': 29705.0}   | {'precision': 0.8511387403281807, 'recall': 0.854637266453459, 'f1-score': 0.8526526632086279, 'support': 29705.0}  |
| No log        | 2.0   | 82   | 0.2777          | {'precision': 0.6244221879815101, 'recall': 0.388915547024952, 'f1-score': 0.4793021880544057, 'support': 4168.0}   | {'precision': 0.7427921092564491, 'recall': 0.9098513011152416, 'f1-score': 0.8178780284043441, 'support': 2152.0} | {'precision': 0.9998232278592893, 'recall': 1.0, 'f1-score': 0.9999116061168567, 'support': 11312.0}                | {'precision': 0.8542442434835473, 'recall': 0.9310858941439576, 'f1-score': 0.8910114140773621, 'support': 12073.0} | 0.8797   | {'precision': 0.8053204421451989, 'recall': 0.8074631855710378, 'f1-score': 0.7970258091632422, 'support': 29705.0} | {'precision': 0.869361097584513, 'recall': 0.8797172193233462, 'f1-score': 0.8694154495030058, 'support': 29705.0}  |
| No log        | 3.0   | 123  | 0.2431          | {'precision': 0.610574601641719, 'recall': 0.6067658349328215, 'f1-score': 0.6086642599277977, 'support': 4168.0}   | {'precision': 0.8699186991869918, 'recall': 0.7955390334572491, 'f1-score': 0.8310679611650487, 'support': 2152.0} | {'precision': 0.9999115435647944, 'recall': 0.9992927864214993, 'f1-score': 0.9996020692399523, 'support': 11312.0} | {'precision': 0.8864117168429617, 'recall': 0.9023440735525553, 'f1-score': 0.8943069408529326, 'support': 12073.0} | 0.8901   | {'precision': 0.8417041403091168, 'recall': 0.8259854320910314, 'f1-score': 0.8334103077964329, 'support': 29705.0} | {'precision': 0.8897353313766411, 'recall': 0.8900521797677159, 'f1-score': 0.8897437196420146, 'support': 29705.0} |
| No log        | 4.0   | 164  | 0.2505          | {'precision': 0.6349911190053286, 'recall': 0.5146353166986565, 'f1-score': 0.5685131195335278, 'support': 4168.0}  | {'precision': 0.8684456928838952, 'recall': 0.8619888475836431, 'f1-score': 0.8652052238805971, 'support': 2152.0} | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8643422891753377, 'recall': 0.921974654187029, 'f1-score': 0.8922287683860366, 'support': 12073.0}  | 0.8902   | {'precision': 0.8419226767953545, 'recall': 0.8246497046173321, 'f1-score': 0.8314757282262835, 'support': 29705.0} | {'precision': 0.8840849237740477, 'recall': 0.8901868372327891, 'f1-score': 0.8858731616505928, 'support': 29705.0} |
| No log        | 5.0   | 205  | 0.2614          | {'precision': 0.6192818494835219, 'recall': 0.6041266794625719, 'f1-score': 0.6116103959193587, 'support': 4168.0}  | {'precision': 0.8873449131513648, 'recall': 0.8308550185873605, 'f1-score': 0.8581713462922966, 'support': 2152.0} | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.884087401510844, 'recall': 0.9015157790110163, 'f1-score': 0.892716535433071, 'support': 12073.0}   | 0.8922   | {'precision': 0.8476564425656468, 'recall': 0.8341243692652373, 'f1-score': 0.8406135196874247, 'support': 29705.0} | {'precision': 0.8912748792655566, 'recall': 0.8921730348426191, 'f1-score': 0.8916089419894234, 'support': 29705.0} |
| No log        | 6.0   | 246  | 0.2767          | {'precision': 0.6314779270633397, 'recall': 0.6314779270633397, 'f1-score': 0.6314779270633397, 'support': 4168.0}  | {'precision': 0.8957934990439771, 'recall': 0.870817843866171, 'f1-score': 0.8831291234684261, 'support': 2152.0}  | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8910319815364326, 'recall': 0.895386399403628, 'f1-score': 0.8932038834951457, 'support': 12073.0}  | 0.8964   | {'precision': 0.8545537534401515, 'recall': 0.8494205425832846, 'f1-score': 0.851941683782971, 'support': 29705.0}  | {'precision': 0.8964206972370266, 'recall': 0.8964147449924256, 'f1-score': 0.8964027733122504, 'support': 29705.0} |
| No log        | 7.0   | 287  | 0.2939          | {'precision': 0.6163162705667276, 'recall': 0.6470729366602687, 'f1-score': 0.6313202247191011, 'support': 4168.0}  | {'precision': 0.8980466888994759, 'recall': 0.8759293680297398, 'f1-score': 0.8868501529051988, 'support': 2152.0} | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8941008643114878, 'recall': 0.8825478340097739, 'f1-score': 0.8882867861609004, 'support': 12073.0} | 0.8938   | {'precision': 0.852093857473637, 'recall': 0.8513875346749455, 'f1-score': 0.8516032412225432, 'support': 29705.0}  | {'precision': 0.895703838190886, 'recall': 0.8937552600572294, 'f1-score': 0.8946517629052753, 'support': 29705.0}  |
| No log        | 8.0   | 328  | 0.3407          | {'precision': 0.6260224948875256, 'recall': 0.5875719769673704, 'f1-score': 0.6061881188118811, 'support': 4168.0}  | {'precision': 0.8996990972918756, 'recall': 0.8336431226765799, 'f1-score': 0.8654124457308249, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0}                | {'precision': 0.8776114624189546, 'recall': 0.9081421353433281, 'f1-score': 0.8926158104697549, 'support': 12073.0} | 0.8925   | {'precision': 0.850833263649589, 'recall': 0.8322067062008507, 'f1-score': 0.8409877748920311, 'support': 29705.0}  | {'precision': 0.8905173338443818, 'recall': 0.8925433428715704, 'f1-score': 0.8912475861436012, 'support': 29705.0} |
| No log        | 9.0   | 369  | 0.3397          | {'precision': 0.6222800378429517, 'recall': 0.6312380038387716, 'f1-score': 0.6267270128632683, 'support': 4168.0}  | {'precision': 0.8995675156174916, 'recall': 0.8698884758364313, 'f1-score': 0.8844790928419561, 'support': 2152.0} | {'precision': 0.9999115983026874, 'recall': 0.9999115983026874, 'f1-score': 0.9999115983026874, 'support': 11312.0} | {'precision': 0.8896888447533929, 'recall': 0.890499461608548, 'f1-score': 0.8900939686219315, 'support': 12073.0}  | 0.8943   | {'precision': 0.852861999129131, 'recall': 0.8478843848966096, 'f1-score': 0.8503029181574608, 'support': 29705.0}  | {'precision': 0.8948576305014636, 'recall': 0.8942938899175223, 'f1-score': 0.8945531621135354, 'support': 29705.0} |
| No log        | 10.0  | 410  | 0.3688          | {'precision': 0.6277702397105382, 'recall': 0.6660268714011516, 'f1-score': 0.6463329452852155, 'support': 4168.0}  | {'precision': 0.8976014760147601, 'recall': 0.904275092936803, 'f1-score': 0.9009259259259259, 'support': 2152.0}  | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.9005253346890357, 'recall': 0.8803114387476186, 'f1-score': 0.890303664921466, 'support': 12073.0}  | 0.8976   | {'precision': 0.8564521641327977, 'recall': 0.8626533507713933, 'f1-score': 0.859379584309395, 'support': 29705.0}  | {'precision': 0.8998898229116948, 'recall': 0.8975593334455478, 'f1-score': 0.8985976932246313, 'support': 29705.0} |
| No log        | 11.0  | 451  | 0.3741          | {'precision': 0.6391675025075225, 'recall': 0.6115642994241842, 'f1-score': 0.6250613045610592, 'support': 4168.0}  | {'precision': 0.8907678244972578, 'recall': 0.9056691449814126, 'f1-score': 0.8981566820276498, 'support': 2152.0} | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8859692206941716, 'recall': 0.8964631823076286, 'f1-score': 0.8911853102227345, 'support': 12073.0} | 0.8966   | {'precision': 0.8539540384539521, 'recall': 0.8534241566783063, 'f1-score': 0.8535897744791039, 'support': 29705.0} | {'precision': 0.8950778992965517, 'recall': 0.8965830668237671, 'f1-score': 0.8957707109763514, 'support': 29705.0} |
| No log        | 12.0  | 492  | 0.4274          | {'precision': 0.6285550129273197, 'recall': 0.5249520153550864, 'f1-score': 0.572100928225912, 'support': 4168.0}   | {'precision': 0.9079259610821072, 'recall': 0.8889405204460966, 'f1-score': 0.8983329420051654, 'support': 2152.0} | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8601999375195252, 'recall': 0.9122836080510229, 'f1-score': 0.8854765445994291, 'support': 12073.0} | 0.8896   | {'precision': 0.8491481294114522, 'recall': 0.8315440359630515, 'f1-score': 0.8389665539838698, 'support': 29705.0} | {'precision': 0.8843584546775584, 'recall': 0.8896482073724962, 'f1-score': 0.8860322338020225, 'support': 29705.0} |
| 0.1761        | 13.0  | 533  | 0.4068          | {'precision': 0.6263554926921263, 'recall': 0.6374760076775432, 'f1-score': 0.6318668252080857, 'support': 4168.0}  | {'precision': 0.8843537414965986, 'recall': 0.9061338289962825, 'f1-score': 0.895111315125086, 'support': 2152.0}  | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8937630807869401, 'recall': 0.8842872525470057, 'f1-score': 0.8889999167291198, 'support': 12073.0} | 0.8953   | {'precision': 0.8510959802731305, 'recall': 0.8569742723052078, 'f1-score': 0.853983464541816, 'support': 29705.0}  | {'precision': 0.8959831916504318, 'recall': 0.8953038209055715, 'f1-score': 0.895616781497613, 'support': 29705.0}  |
| 0.1761        | 14.0  | 574  | 0.4170          | {'precision': 0.6411675893306492, 'recall': 0.6113243761996161, 'f1-score': 0.6258904446082044, 'support': 4168.0}  | {'precision': 0.918905715681485, 'recall': 0.8740706319702602, 'f1-score': 0.8959276018099548, 'support': 2152.0}  | {'precision': 0.9999116061168567, 'recall': 1.0, 'f1-score': 0.9999558011049724, 'support': 11312.0}                | {'precision': 0.8821437232236683, 'recall': 0.9039178331814793, 'f1-score': 0.8928980526918672, 'support': 12073.0} | 0.8973   | {'precision': 0.8605321585881648, 'recall': 0.8473282103378389, 'f1-score': 0.8536679750537497, 'support': 29705.0} | {'precision': 0.8958422107843774, 'recall': 0.8972900185154015, 'f1-score': 0.8964216725962088, 'support': 29705.0} |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2