---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: sep_tok
      split: train[40%:60%]
      args: sep_tok
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.9137861922234899
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-sep_tok

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4085
- Claim: {'precision': 0.6911519198664441, 'recall': 0.7267939433838051, 'f1-score': 0.708524975933255, 'support': 4557.0}
- Majorclaim: {'precision': 0.8940252943741823, 'recall': 0.9034817100044072, 'f1-score': 0.8987286277948269, 'support': 2269.0}
- O: {'precision': 0.9999095350099512, 'recall': 0.9983741306115076, 'f1-score': 0.9991412429378531, 'support': 11071.0}
- Premise: {'precision': 0.9249930030786454, 'recall': 0.9095913031512316, 'f1-score': 0.9172275029487268, 'support': 14534.0}
- Accuracy: 0.9138
- Macro avg: {'precision': 0.8775199380823058, 'recall': 0.8845602717877379, 'f1-score': 0.8809055874036654, 'support': 32431.0}
- Weighted avg: {'precision': 0.9155428281769482, 'recall': 0.9137861922234899, 'f1-score': 0.914570651543772, 'support': 32431.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 16

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                              | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.3528          | {'precision': 0.5627967198964178, 'recall': 0.5723063418915953, 'f1-score': 0.5675116962245675, 'support': 4557.0} | {'precision': 0.7071971390254805, 'recall': 0.6972234464521816, 'f1-score': 0.7021748779405237, 'support': 2269.0} | {'precision': 0.9985252096967463, 'recall': 0.9785023936410442, 'f1-score': 0.9884124087591241, 'support': 11071.0} | {'precision': 0.8892665352457345, 'recall': 0.900096325856612, 'f1-score': 0.8946486578902376, 'support': 14534.0}  | 0.8666   | {'precision': 0.7894464009660948, 'recall': 0.7870321269603583, 'f1-score': 0.7881869102036132, 'support': 32431.0} | {'precision': 0.8679524954775053, 'recall': 0.8666091085689618, 'f1-score': 0.8672234272421873, 'support': 32431.0} |
| No log        | 2.0   | 82   | 0.2667          | {'precision': 0.7048665620094191, 'recall': 0.4926486723721747, 'f1-score': 0.5799535003874967, 'support': 4557.0} | {'precision': 0.8561310314298363, 'recall': 0.8523578669017188, 'f1-score': 0.8542402826855124, 'support': 2269.0} | {'precision': 0.9996358670914884, 'recall': 0.9918706530575377, 'f1-score': 0.9957381211461733, 'support': 11071.0} | {'precision': 0.8665166854143233, 'recall': 0.954038805559378, 'f1-score': 0.9081739586062353, 'support': 14534.0}  | 0.8950   | {'precision': 0.8567875364862668, 'recall': 0.8227289994727023, 'f1-score': 0.8345264657063545, 'support': 32431.0} | {'precision': 0.8885190226564973, 'recall': 0.8950078628472757, 'f1-score': 0.8881729319562012, 'support': 32431.0} |
| No log        | 3.0   | 123  | 0.2512          | {'precision': 0.6279952076677316, 'recall': 0.6901470265525566, 'f1-score': 0.6576058546785155, 'support': 4557.0} | {'precision': 0.7716945996275605, 'recall': 0.9131776112825033, 'f1-score': 0.836495761001211, 'support': 2269.0}  | {'precision': 0.9997280638143582, 'recall': 0.9962063047601842, 'f1-score': 0.997964077274578, 'support': 11071.0}  | {'precision': 0.9338246023639282, 'recall': 0.8806247419843126, 'f1-score': 0.9064447592067989, 'support': 14534.0} | 0.8956   | {'precision': 0.8333106183683946, 'recall': 0.8700389211448892, 'f1-score': 0.8496276130402758, 'support': 32431.0} | {'precision': 0.9020056542549684, 'recall': 0.8955937220560575, 'f1-score': 0.8978276091178259, 'support': 32431.0} |
| No log        | 4.0   | 164  | 0.2493          | {'precision': 0.5918265221017515, 'recall': 0.7785824007022164, 'f1-score': 0.6724791508718727, 'support': 4557.0} | {'precision': 0.9070698088039129, 'recall': 0.8990744821507272, 'f1-score': 0.9030544488711819, 'support': 2269.0} | {'precision': 1.0, 'recall': 0.9983741306115076, 'f1-score': 0.9991864039052614, 'support': 11071.0}                | {'precision': 0.9359677173747526, 'recall': 0.8458098252373745, 'f1-score': 0.8886077779384126, 'support': 14534.0} | 0.8922   | {'precision': 0.8587160120701043, 'recall': 0.8804602096754564, 'f1-score': 0.8658319453966822, 'support': 32431.0} | {'precision': 0.907448110194518, 'recall': 0.8921710708889643, 'f1-score': 0.8969978155839744, 'support': 32431.0}  |
| No log        | 5.0   | 205  | 0.2277          | {'precision': 0.7054698457223001, 'recall': 0.6622778143515471, 'f1-score': 0.6831918505942275, 'support': 4557.0} | {'precision': 0.9207650273224044, 'recall': 0.8911414720141031, 'f1-score': 0.9057110862262038, 'support': 2269.0} | {'precision': 1.0, 'recall': 0.9978321741486768, 'f1-score': 0.9989149109322725, 'support': 11071.0}                | {'precision': 0.9053655264922871, 'recall': 0.9287876702903537, 'f1-score': 0.9169270479554409, 'support': 14534.0} | 0.9123   | {'precision': 0.8829000998842478, 'recall': 0.8700097827011701, 'f1-score': 0.8761862239270362, 'support': 32431.0} | {'precision': 0.9106603094566913, 'recall': 0.912275292158737, 'f1-score': 0.9112876078974042, 'support': 32431.0}  |
| No log        | 6.0   | 246  | 0.2558          | {'precision': 0.6772561715904493, 'recall': 0.7344744349352644, 'f1-score': 0.7047057585008949, 'support': 4557.0} | {'precision': 0.9172320217096337, 'recall': 0.8937858087263112, 'f1-score': 0.9053571428571427, 'support': 2269.0} | {'precision': 0.9999095268252963, 'recall': 0.998283804534369, 'f1-score': 0.9990960043391791, 'support': 11071.0}  | {'precision': 0.9226713532513181, 'recall': 0.9030549057382689, 'f1-score': 0.9127577454014394, 'support': 14534.0} | 0.9112   | {'precision': 0.8792672683441743, 'recall': 0.8823997384835534, 'f1-score': 0.880479162774664, 'support': 32431.0}  | {'precision': 0.9141734652287734, 'recall': 0.9112269125219697, 'f1-score': 0.9124791845559805, 'support': 32431.0} |
| No log        | 7.0   | 287  | 0.2902          | {'precision': 0.7137206427688504, 'recall': 0.6335308316875137, 'f1-score': 0.671239246686817, 'support': 4557.0}  | {'precision': 0.9082039911308204, 'recall': 0.9026002644336713, 'f1-score': 0.9053934571175951, 'support': 2269.0} | {'precision': 0.9999095431931253, 'recall': 0.998464456688646, 'f1-score': 0.999186477447347, 'support': 11071.0}   | {'precision': 0.8983152029716105, 'recall': 0.9318150543553048, 'f1-score': 0.9147585275244849, 'support': 14534.0} | 0.9106   | {'precision': 0.8800373450161016, 'recall': 0.8666026517912839, 'f1-score': 0.872644427194061, 'support': 32431.0}  | {'precision': 0.9077503480513693, 'recall': 0.910610218617989, 'f1-score': 0.9087067599584377, 'support': 32431.0}  |
| No log        | 8.0   | 328  | 0.3177          | {'precision': 0.7160274320548641, 'recall': 0.618608733816107, 'f1-score': 0.6637626559924652, 'support': 4557.0}  | {'precision': 0.8826768318509106, 'recall': 0.9184662847069194, 'f1-score': 0.9002159827213824, 'support': 2269.0} | {'precision': 0.9999095431931253, 'recall': 0.998464456688646, 'f1-score': 0.999186477447347, 'support': 11071.0}   | {'precision': 0.8981960472211169, 'recall': 0.9318150543553048, 'f1-score': 0.914696744563015, 'support': 14534.0}  | 0.9096   | {'precision': 0.8742024635800042, 'recall': 0.8668386323917443, 'f1-score': 0.8694654651810524, 'support': 32431.0} | {'precision': 0.906235103522757, 'recall': 0.9096235083716198, 'f1-score': 0.9072662719450809, 'support': 32431.0}  |
| No log        | 9.0   | 369  | 0.3188          | {'precision': 0.6808467741935483, 'recall': 0.7410577134079438, 'f1-score': 0.7096774193548386, 'support': 4557.0} | {'precision': 0.9019170753455193, 'recall': 0.8915821947994711, 'f1-score': 0.8967198581560283, 'support': 2269.0} | {'precision': 0.999909600433918, 'recall': 0.9990967392286153, 'f1-score': 0.9995030045633218, 'support': 11071.0}  | {'precision': 0.927996611605252, 'recall': 0.9044997935874501, 'f1-score': 0.9160975609756098, 'support': 14534.0}  | 0.9129   | {'precision': 0.8776675153945594, 'recall': 0.8840591102558701, 'f1-score': 0.8804994607624497, 'support': 32431.0} | {'precision': 0.9159930478071482, 'recall': 0.9129228207579168, 'f1-score': 0.9142091539852634, 'support': 32431.0} |
| No log        | 10.0  | 410  | 0.3553          | {'precision': 0.6544131725062224, 'recall': 0.750054860653939, 'f1-score': 0.6989775051124745, 'support': 4557.0}  | {'precision': 0.8812341504649197, 'recall': 0.9189070074922874, 'f1-score': 0.8996763754045308, 'support': 2269.0} | {'precision': 0.9999095677337674, 'recall': 0.9987354349200614, 'f1-score': 0.9993221564462921, 'support': 11071.0} | {'precision': 0.933473592571097, 'recall': 0.885303426448328, 'f1-score': 0.9087506179814958, 'support': 14534.0}   | 0.9074   | {'precision': 0.8672576208190016, 'recall': 0.888250182378654, 'f1-score': 0.8766816637361983, 'support': 32431.0}  | {'precision': 0.9132862117518615, 'recall': 0.90737257562209, 'f1-score': 0.9095582394113776, 'support': 32431.0}   |
| No log        | 11.0  | 451  | 0.3296          | {'precision': 0.6971476510067114, 'recall': 0.7294272547728768, 'f1-score': 0.7129222520107238, 'support': 4557.0} | {'precision': 0.8972542072630647, 'recall': 0.8929043631555752, 'f1-score': 0.8950740004417939, 'support': 2269.0} | {'precision': 0.999909412084428, 'recall': 0.9970192394544305, 'f1-score': 0.9984622342831299, 'support': 11071.0}  | {'precision': 0.9236391479883057, 'recall': 0.9129627081326545, 'f1-score': 0.9182698961937715, 'support': 14534.0} | 0.9145   | {'precision': 0.8794876045856275, 'recall': 0.8830783913788842, 'f1-score': 0.8811820957323547, 'support': 32431.0} | {'precision': 0.9160044438952304, 'recall': 0.9144645555178688, 'f1-score': 0.9151681932855632, 'support': 32431.0} |
| No log        | 12.0  | 492  | 0.3576          | {'precision': 0.7129815185927411, 'recall': 0.7026552556506473, 'f1-score': 0.7077807250221043, 'support': 4557.0} | {'precision': 0.9037336932073774, 'recall': 0.8854120758043191, 'f1-score': 0.8944790739091719, 'support': 2269.0} | {'precision': 0.9995480021695896, 'recall': 0.9987354349200614, 'f1-score': 0.9991415533366468, 'support': 11071.0} | {'precision': 0.9171613783691572, 'recall': 0.9247970276592817, 'f1-score': 0.9209633766144781, 'support': 14534.0} | 0.9161   | {'precision': 0.8833561480847163, 'recall': 0.8778999485085774, 'f1-score': 0.8805911822206004, 'support': 32431.0} | {'precision': 0.9156562528245048, 'recall': 0.9160679596682186, 'f1-score': 0.9158431018263538, 'support': 32431.0} |
| 0.1716        | 13.0  | 533  | 0.3893          | {'precision': 0.7014310494362532, 'recall': 0.7098968619705947, 'f1-score': 0.7056385647289781, 'support': 4557.0} | {'precision': 0.9027531083481349, 'recall': 0.8959894226531512, 'f1-score': 0.8993585489935855, 'support': 2269.0} | {'precision': 0.9999095677337674, 'recall': 0.9987354349200614, 'f1-score': 0.9993221564462921, 'support': 11071.0} | {'precision': 0.9189468605693019, 'recall': 0.9173661758634925, 'f1-score': 0.9181558378955342, 'support': 14534.0} | 0.9145   | {'precision': 0.8807601465218644, 'recall': 0.880496973851825, 'f1-score': 0.8806187770160975, 'support': 32431.0}  | {'precision': 0.914888242453754, 'recall': 0.9144953902130677, 'f1-score': 0.9146869362377662, 'support': 32431.0}  |
| 0.1716        | 14.0  | 574  | 0.4032          | {'precision': 0.6816792337477073, 'recall': 0.7340355497037525, 'f1-score': 0.7068892645815723, 'support': 4557.0} | {'precision': 0.8945295404814004, 'recall': 0.9008373732921993, 'f1-score': 0.8976723759332456, 'support': 2269.0} | {'precision': 0.9999095431931253, 'recall': 0.998464456688646, 'f1-score': 0.999186477447347, 'support': 11071.0}   | {'precision': 0.9270304568527918, 'recall': 0.9047062061373331, 'f1-score': 0.9157322933351906, 'support': 14534.0} | 0.9125   | {'precision': 0.8757871935687562, 'recall': 0.8845108964554826, 'f1-score': 0.8798701028243389, 'support': 32431.0} | {'precision': 0.915160155657555, 'recall': 0.9124603003299312, 'f1-score': 0.9136122735297709, 'support': 32431.0}  |
| 0.1716        | 15.0  | 615  | 0.4182          | {'precision': 0.6571861964353432, 'recall': 0.760588106210226, 'f1-score': 0.7051164683145152, 'support': 4557.0}  | {'precision': 0.8850574712643678, 'recall': 0.9162626707800793, 'f1-score': 0.9003897791251624, 'support': 2269.0} | {'precision': 0.9999095268252963, 'recall': 0.998283804534369, 'f1-score': 0.9990960043391791, 'support': 11071.0}  | {'precision': 0.9354416575790622, 'recall': 0.885303426448328, 'f1-score': 0.9096822086323306, 'support': 14534.0}  | 0.9085   | {'precision': 0.8693987130260175, 'recall': 0.8901095019932507, 'f1-score': 0.8785711151027968, 'support': 32431.0} | {'precision': 0.9148253313863789, 'recall': 0.9085134593444544, 'f1-score': 0.910811052364885, 'support': 32431.0}  |
| 0.1716        | 16.0  | 656  | 0.4085          | {'precision': 0.6911519198664441, 'recall': 0.7267939433838051, 'f1-score': 0.708524975933255, 'support': 4557.0}  | {'precision': 0.8940252943741823, 'recall': 0.9034817100044072, 'f1-score': 0.8987286277948269, 'support': 2269.0} | {'precision': 0.9999095350099512, 'recall': 0.9983741306115076, 'f1-score': 0.9991412429378531, 'support': 11071.0} | {'precision': 0.9249930030786454, 'recall': 0.9095913031512316, 'f1-score': 0.9172275029487268, 'support': 14534.0} | 0.9138   | {'precision': 0.8775199380823058, 'recall': 0.8845602717877379, 'f1-score': 0.8809055874036654, 'support': 32431.0} | {'precision': 0.9155428281769482, 'recall': 0.9137861922234899, 'f1-score': 0.914570651543772, 'support': 32431.0}  |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2