---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: sep_tok
      split: train[0%:20%]
      args: sep_tok
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.9033544303797468
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-sep_tok

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4725
- Claim: {'precision': 0.6448837209302326, 'recall': 0.6506335053965274, 'f1-score': 0.6477458537724831, 'support': 4262.0}
- Majorclaim: {'precision': 0.9238191975622143, 'recall': 0.8401847575057737, 'f1-score': 0.8800193517174649, 'support': 2165.0}
- O: {'precision': 0.9978613144690301, 'recall': 0.9997527608373167, 'f1-score': 0.9988061421925816, 'support': 12134.0}
- Premise: {'precision': 0.8974495217853348, 'recall': 0.9067413145179845, 'f1-score': 0.9020714912448022, 'support': 13039.0}
- Accuracy: 0.9034
- Macro avg: {'precision': 0.866003438686703, 'recall': 0.8493280845644006, 'f1-score': 0.857160709731833, 'support': 31600.0}
- Weighted avg: {'precision': 0.9037486229637037, 'recall': 0.9033544303797468, 'f1-score': 0.9034037540807719, 'support': 31600.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 16

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.3643          | {'precision': 0.4827682824320538, 'recall': 0.4042702956358517, 'f1-score': 0.44004597114033966, 'support': 4262.0} | {'precision': 0.7040155440414507, 'recall': 0.5020785219399538, 'f1-score': 0.586141817201402, 'support': 2165.0}  | {'precision': 0.9970286102385602, 'recall': 0.967858908851162, 'f1-score': 0.9822272404131643, 'support': 12134.0}  | {'precision': 0.8283247212401414, 'recall': 0.9343507937725286, 'f1-score': 0.8781489890799006, 'support': 13039.0} | 0.8461   | {'precision': 0.7530342894880515, 'recall': 0.7021396300498741, 'f1-score': 0.7216410044587016, 'support': 31600.0} | {'precision': 0.8379817490335457, 'recall': 0.8461075949367088, 'f1-score': 0.8390190812350419, 'support': 31600.0} |
| No log        | 2.0   | 82   | 0.2769          | {'precision': 0.6401985111662531, 'recall': 0.4237447207883623, 'f1-score': 0.5099534095722152, 'support': 4262.0}  | {'precision': 0.6977007161703732, 'recall': 0.8549653579676675, 'f1-score': 0.7683686176836863, 'support': 2165.0} | {'precision': 0.9958073002301875, 'recall': 0.9982693258612164, 'f1-score': 0.9970367931516998, 'support': 12134.0} | {'precision': 0.8682137229623264, 'recall': 0.9296725208988419, 'f1-score': 0.8978926706418281, 'support': 13039.0} | 0.8827   | {'precision': 0.800480062632285, 'recall': 0.801662981379022, 'f1-score': 0.7933128727623573, 'support': 31600.0}   | {'precision': 0.8747725512594399, 'recall': 0.8826582278481012, 'f1-score': 0.8747660275153002, 'support': 31600.0} |
| No log        | 3.0   | 123  | 0.2435          | {'precision': 0.6495901639344263, 'recall': 0.5950258094791178, 'f1-score': 0.621111927504286, 'support': 4262.0}   | {'precision': 0.798990748528175, 'recall': 0.8775981524249422, 'f1-score': 0.836451683909311, 'support': 2165.0}   | {'precision': 0.9976969896364534, 'recall': 0.9996703477830888, 'f1-score': 0.9986826938909928, 'support': 12134.0} | {'precision': 0.9022796352583586, 'recall': 0.9106526574123782, 'f1-score': 0.9064468109469827, 'support': 13039.0} | 0.9      | {'precision': 0.8371393843393533, 'recall': 0.8457367417748817, 'f1-score': 0.8406732790628931, 'support': 31600.0} | {'precision': 0.8977610027099522, 'recall': 0.9, 'f1-score': 0.8985845793132259, 'support': 31600.0}                |
| No log        | 4.0   | 164  | 0.2710          | {'precision': 0.5950994175537256, 'recall': 0.6952135147817926, 'f1-score': 0.6412725895465858, 'support': 4262.0}  | {'precision': 0.781058282208589, 'recall': 0.9408775981524249, 'f1-score': 0.8535512256442489, 'support': 2165.0}  | {'precision': 0.9986824769433466, 'recall': 0.9995055216746332, 'f1-score': 0.9990938298047615, 'support': 12134.0} | {'precision': 0.9320077512848597, 'recall': 0.8483779430937956, 'f1-score': 0.8882286815480969, 'support': 13039.0} | 0.8921   | {'precision': 0.8267119819976302, 'recall': 0.8709936444256616, 'f1-score': 0.8455365816359233, 'support': 31600.0} | {'precision': 0.9018280741401717, 'recall': 0.8920886075949367, 'f1-score': 0.8951158382824038, 'support': 31600.0} |
| No log        | 5.0   | 205  | 0.3124          | {'precision': 0.6569706498951782, 'recall': 0.5882214922571563, 'f1-score': 0.6206981926219362, 'support': 4262.0}  | {'precision': 0.9552845528455285, 'recall': 0.7598152424942263, 'f1-score': 0.8464111139696424, 'support': 2165.0} | {'precision': 0.9973686374475783, 'recall': 0.9995879347288611, 'f1-score': 0.9984770528915414, 'support': 12134.0} | {'precision': 0.877131141644486, 'recall': 0.9351177237518215, 'f1-score': 0.9051967334818115, 'support': 13039.0}  | 0.9011   | {'precision': 0.8716887454581927, 'recall': 0.8206855983080162, 'f1-score': 0.8426957732412328, 'support': 31600.0} | {'precision': 0.8989615180207339, 'recall': 0.9010759493670886, 'f1-score': 0.8986163457707048, 'support': 31600.0} |
| No log        | 6.0   | 246  | 0.3000          | {'precision': 0.6428049671292915, 'recall': 0.6194274988268419, 'f1-score': 0.6308997490739635, 'support': 4262.0}  | {'precision': 0.9450247388675096, 'recall': 0.7939953810623557, 'f1-score': 0.8629518072289157, 'support': 2165.0} | {'precision': 0.996794608366894, 'recall': 0.9995055216746332, 'f1-score': 0.9981482243529073, 'support': 12134.0}  | {'precision': 0.8865773302731916, 'recall': 0.9183986502032364, 'f1-score': 0.9022074888872147, 'support': 13039.0} | 0.9007   | {'precision': 0.8678004111592217, 'recall': 0.8328317629417669, 'f1-score': 0.8485518173857503, 'support': 31600.0} | {'precision': 0.9000253454718111, 'recall': 0.9006962025316456, 'f1-score': 0.8997658036424813, 'support': 31600.0} |
| No log        | 7.0   | 287  | 0.3219          | {'precision': 0.5949652067130577, 'recall': 0.6820741435945565, 'f1-score': 0.6355487538259729, 'support': 4262.0}  | {'precision': 0.9873817034700315, 'recall': 0.7228637413394919, 'f1-score': 0.8346666666666667, 'support': 2165.0} | {'precision': 0.9974464579901153, 'recall': 0.9979396736443052, 'f1-score': 0.9976930048611683, 'support': 12134.0} | {'precision': 0.8966048194626223, 'recall': 0.8931666538845003, 'f1-score': 0.8948824343015215, 'support': 13039.0} | 0.8933   | {'precision': 0.8690995469089566, 'recall': 0.8240110531157135, 'f1-score': 0.8406977149138323, 'support': 31600.0} | {'precision': 0.900862932318002, 'recall': 0.8932594936708861, 'f1-score': 0.8952576298728666, 'support': 31600.0}  |
| No log        | 8.0   | 328  | 0.3370          | {'precision': 0.6643180674383493, 'recall': 0.6194274988268419, 'f1-score': 0.6410879067508499, 'support': 4262.0}  | {'precision': 0.9216867469879518, 'recall': 0.8480369515011548, 'f1-score': 0.8833293240317538, 'support': 2165.0} | {'precision': 0.9963854431939538, 'recall': 0.9995879347288611, 'f1-score': 0.9979841198008804, 'support': 12134.0} | {'precision': 0.889755590223609, 'recall': 0.918552036199095, 'f1-score': 0.9039245283018866, 'support': 13039.0}   | 0.9045   | {'precision': 0.868036461960966, 'recall': 0.8464011053139882, 'f1-score': 0.8565814697213427, 'support': 31600.0}  | {'precision': 0.9024822632687415, 'recall': 0.9044936708860759, 'f1-score': 0.9031815151675017, 'support': 31600.0} |
| No log        | 9.0   | 369  | 0.3583          | {'precision': 0.6696588868940754, 'recall': 0.6126231816048804, 'f1-score': 0.6398725646366867, 'support': 4262.0}  | {'precision': 0.923741738688358, 'recall': 0.8392609699769054, 'f1-score': 0.8794772507260408, 'support': 2165.0}  | {'precision': 0.9980212713331684, 'recall': 0.9976100214273941, 'f1-score': 0.9978156040060999, 'support': 12134.0} | {'precision': 0.8865123116501287, 'recall': 0.9249942480251553, 'f1-score': 0.9053445428614323, 'support': 13039.0} | 0.9049   | {'precision': 0.8694835521414326, 'recall': 0.8436221052585838, 'f1-score': 0.8556274905575649, 'support': 31600.0} | {'precision': 0.9026332651318208, 'recall': 0.904873417721519, 'f1-score': 0.9032749098634072, 'support': 31600.0}  |
| No log        | 10.0  | 410  | 0.4008          | {'precision': 0.6432761242667825, 'recall': 0.6947442515251055, 'f1-score': 0.6680203045685279, 'support': 4262.0}  | {'precision': 0.8905405405405405, 'recall': 0.9131639722863741, 'f1-score': 0.9017103762827823, 'support': 2165.0} | {'precision': 0.9983533673637411, 'recall': 0.9993406955661777, 'f1-score': 0.9988467874794069, 'support': 12134.0} | {'precision': 0.9138627187079408, 'recall': 0.8852672750977836, 'f1-score': 0.8993377483443709, 'support': 13039.0} | 0.9053   | {'precision': 0.8615081877197512, 'recall': 0.8731290486188602, 'f1-score': 0.866978804168772, 'support': 31600.0}  | {'precision': 0.9082132550860689, 'recall': 0.9052848101265822, 'f1-score': 0.9065119405905274, 'support': 31600.0} |
| No log        | 11.0  | 451  | 0.4241          | {'precision': 0.6283344541582605, 'recall': 0.6576724542468325, 'f1-score': 0.6426688066032329, 'support': 4262.0}  | {'precision': 0.9258312020460358, 'recall': 0.836027713625866, 'f1-score': 0.8786407766990292, 'support': 2165.0}  | {'precision': 0.9981066842278564, 'recall': 0.9992582825119499, 'f1-score': 0.9986821513878593, 'support': 12134.0} | {'precision': 0.8980515495550783, 'recall': 0.897844926758187, 'f1-score': 0.897948226270374, 'support': 13039.0}   | 0.9002   | {'precision': 0.8625809724968077, 'recall': 0.8477008442857088, 'f1-score': 0.8544849902401238, 'support': 31600.0} | {'precision': 0.9019970461114446, 'recall': 0.9001582278481013, 'f1-score': 0.900875565904306, 'support': 31600.0}  |
| No log        | 12.0  | 492  | 0.4143          | {'precision': 0.6269706007669366, 'recall': 0.6905208822149226, 'f1-score': 0.6572130415364003, 'support': 4262.0}  | {'precision': 0.9402756508422665, 'recall': 0.8508083140877598, 'f1-score': 0.8933074684772067, 'support': 2165.0} | {'precision': 0.998682042833608, 'recall': 0.9991758694577221, 'f1-score': 0.9989288951141139, 'support': 12134.0}  | {'precision': 0.9047395955336925, 'recall': 0.8886417670066723, 'f1-score': 0.8966184322525729, 'support': 13039.0} | 0.9018   | {'precision': 0.8676669724941259, 'recall': 0.8572867081917691, 'f1-score': 0.8615169593450734, 'support': 31600.0} | {'precision': 0.9057833221028166, 'recall': 0.9017721518987342, 'f1-score': 0.9033880887258622, 'support': 31600.0} |
| 0.1652        | 13.0  | 533  | 0.4456          | {'precision': 0.6519438693351737, 'recall': 0.664946034725481, 'f1-score': 0.6583807643164131, 'support': 4262.0}   | {'precision': 0.9468421052631579, 'recall': 0.8309468822170901, 'f1-score': 0.8851168511685117, 'support': 2165.0} | {'precision': 0.9975333004440059, 'recall': 0.9998351738915444, 'f1-score': 0.9986829107672046, 'support': 12134.0} | {'precision': 0.8987946327041164, 'recall': 0.909272183449651, 'f1-score': 0.9040030499428136, 'support': 13039.0}  | 0.9057   | {'precision': 0.8737784769366135, 'recall': 0.8512500685709417, 'f1-score': 0.8615458940487357, 'support': 31600.0} | {'precision': 0.9067072852030946, 'recall': 0.9057278481012658, 'f1-score': 0.905937057207278, 'support': 31600.0}  |
| 0.1652        | 14.0  | 574  | 0.4601          | {'precision': 0.6456975960458324, 'recall': 0.674331299859221, 'f1-score': 0.6597038907379777, 'support': 4262.0}   | {'precision': 0.9215880893300248, 'recall': 0.8577367205542725, 'f1-score': 0.8885167464114833, 'support': 2165.0} | {'precision': 0.9972053263192504, 'recall': 0.9998351738915444, 'f1-score': 0.9985185185185186, 'support': 12134.0} | {'precision': 0.9043028994447871, 'recall': 0.8993787867167727, 'f1-score': 0.9018341215826509, 'support': 13039.0} | 0.9047   | {'precision': 0.8671984777849736, 'recall': 0.8578204952554527, 'f1-score': 0.8621433193126575, 'support': 31600.0} | {'precision': 0.9062815285811774, 'recall': 0.904746835443038, 'f1-score': 0.9053903656115826, 'support': 31600.0}  |
| 0.1652        | 15.0  | 615  | 0.4733          | {'precision': 0.6527744124061061, 'recall': 0.6320976067573909, 'f1-score': 0.6422696388127309, 'support': 4262.0}  | {'precision': 0.9225352112676056, 'recall': 0.8471131639722864, 'f1-score': 0.883216951601252, 'support': 2165.0}  | {'precision': 0.9979432332373509, 'recall': 0.9996703477830888, 'f1-score': 0.9988060438881798, 'support': 12134.0} | {'precision': 0.8922730682670668, 'recall': 0.912186517370964, 'f1-score': 0.9021199135348325, 'support': 13039.0}  | 0.9035   | {'precision': 0.8663814812945323, 'recall': 0.8477669089709325, 'f1-score': 0.8566031369592488, 'support': 31600.0} | {'precision': 0.9026204116235915, 'recall': 0.9035443037974683, 'f1-score': 0.9029041768973552, 'support': 31600.0} |
| 0.1652        | 16.0  | 656  | 0.4725          | {'precision': 0.6448837209302326, 'recall': 0.6506335053965274, 'f1-score': 0.6477458537724831, 'support': 4262.0}  | {'precision': 0.9238191975622143, 'recall': 0.8401847575057737, 'f1-score': 0.8800193517174649, 'support': 2165.0} | {'precision': 0.9978613144690301, 'recall': 0.9997527608373167, 'f1-score': 0.9988061421925816, 'support': 12134.0} | {'precision': 0.8974495217853348, 'recall': 0.9067413145179845, 'f1-score': 0.9020714912448022, 'support': 13039.0} | 0.9034   | {'precision': 0.866003438686703, 'recall': 0.8493280845644006, 'f1-score': 0.857160709731833, 'support': 31600.0}   | {'precision': 0.9037486229637037, 'recall': 0.9033544303797468, 'f1-score': 0.9034037540807719, 'support': 31600.0} |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2