meta_data/README_s42_e15.md

---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: sep_tok
      split: train[80%:100%]
      args: sep_tok
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8996128597879145
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-sep_tok

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4440
- Claim: {'precision': 0.6588266384778013, 'recall': 0.5981285988483686, 'f1-score': 0.6270120724346077, 'support': 4168.0}
- Majorclaim: {'precision': 0.9058606368251039, 'recall': 0.9121747211895911, 'f1-score': 0.9090067145172495, 'support': 2152.0}
- O: {'precision': 1.0, 'recall': 0.999557991513437, 'f1-score': 0.9997789469030461, 'support': 11312.0}
- Premise: {'precision': 0.8805334618783642, 'recall': 0.9078108175267124, 'f1-score': 0.8939641109298531, 'support': 12073.0}
- Accuracy: 0.8996
- Macro avg: {'precision': 0.8613051842953173, 'recall': 0.8544180322695273, 'f1-score': 0.857440461196189, 'support': 29705.0}
- Weighted avg: {'precision': 0.8967541492974446, 'recall': 0.8996128597879145, 'f1-score': 0.8978925071931304, 'support': 29705.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.3459          | {'precision': 0.5076716016150741, 'recall': 0.45249520153550865, 'f1-score': 0.4784980337434987, 'support': 4168.0} | {'precision': 0.6691762621789193, 'recall': 0.7021375464684015, 'f1-score': 0.6852607709750567, 'support': 2152.0} | {'precision': 0.9995503597122302, 'recall': 0.98258486562942, 'f1-score': 0.9909950071326675, 'support': 11312.0}   | {'precision': 0.8565651760228354, 'recall': 0.8948065932245507, 'f1-score': 0.8752683816082643, 'support': 12073.0} | 0.8522   | {'precision': 0.7582408498822648, 'recall': 0.7580060517144703, 'f1-score': 0.7575055483648718, 'support': 29705.0} | {'precision': 0.8484856957054067, 'recall': 0.8522134320821411, 'f1-score': 0.8499010831719419, 'support': 29705.0} |
| No log        | 2.0   | 82   | 0.2620          | {'precision': 0.6607431340872375, 'recall': 0.3925143953934741, 'f1-score': 0.49247441300421435, 'support': 4168.0} | {'precision': 0.7220035778175313, 'recall': 0.9377323420074349, 'f1-score': 0.815847988680008, 'support': 2152.0}  | {'precision': 0.9998230714791224, 'recall': 0.9991159830268741, 'f1-score': 0.9994694021931375, 'support': 11312.0} | {'precision': 0.8597867479055598, 'recall': 0.9350617079433446, 'f1-score': 0.8958457326508749, 'support': 12073.0} | 0.8835   | {'precision': 0.8105891328223628, 'recall': 0.8161061070927819, 'f1-score': 0.8009093841320587, 'support': 29705.0} | {'precision': 0.8752039412346267, 'recall': 0.8835212927116647, 'f1-score': 0.8729130325852121, 'support': 29705.0} |
| No log        | 3.0   | 123  | 0.2374          | {'precision': 0.6255685898970553, 'recall': 0.6269193857965452, 'f1-score': 0.6262432594367885, 'support': 4168.0}  | {'precision': 0.8762836185819071, 'recall': 0.8327137546468402, 'f1-score': 0.8539432928282107, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9992927864214993, 'f1-score': 0.9996462681287585, 'support': 11312.0}                | {'precision': 0.8920272600377699, 'recall': 0.8998591899279383, 'f1-score': 0.8959261091868711, 'support': 12073.0} | 0.8946   | {'precision': 0.848469867129183, 'recall': 0.8396962791982058, 'f1-score': 0.8439397323951572, 'support': 29705.0}  | {'precision': 0.894616305009769, 'recall': 0.8945632048476687, 'f1-score': 0.8945424128188674, 'support': 29705.0}  |
| No log        | 4.0   | 164  | 0.2502          | {'precision': 0.6409472880061116, 'recall': 0.6038867562380038, 'f1-score': 0.6218653489808523, 'support': 4168.0}  | {'precision': 0.8598290598290599, 'recall': 0.9349442379182156, 'f1-score': 0.8958147818343721, 'support': 2152.0} | {'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0} | {'precision': 0.8900247320692498, 'recall': 0.8942267870454734, 'f1-score': 0.8921208114696525, 'support': 12073.0} | 0.8965   | {'precision': 0.8476560534922002, 'recall': 0.8581318427544543, 'f1-score': 0.8523618182414225, 'support': 29705.0} | {'precision': 0.8947008354139008, 'recall': 0.8965157380912304, 'f1-score': 0.8954149818075824, 'support': 29705.0} |
| No log        | 5.0   | 205  | 0.2594          | {'precision': 0.6565992865636148, 'recall': 0.6624280230326296, 'f1-score': 0.6595007763047892, 'support': 4168.0}  | {'precision': 0.8777137793531237, 'recall': 0.9205390334572491, 'f1-score': 0.8986164663188932, 'support': 2152.0} | {'precision': 0.9999115670321896, 'recall': 0.999557991513437, 'f1-score': 0.9997347480106101, 'support': 11312.0}  | {'precision': 0.9036447423544198, 'recall': 0.8933156630497805, 'f1-score': 0.8984505164945018, 'support': 12073.0} | 0.9033   | {'precision': 0.8594673438258369, 'recall': 0.868960177763274, 'f1-score': 0.8640756267821986, 'support': 29705.0}  | {'precision': 0.903761942443296, 'recall': 0.9033496044436964, 'f1-score': 0.9035049461804666, 'support': 29705.0}  |
| No log        | 6.0   | 246  | 0.2753          | {'precision': 0.637114951164538, 'recall': 0.6103646833013435, 'f1-score': 0.6234530082097782, 'support': 4168.0}   | {'precision': 0.9366306027820711, 'recall': 0.8447955390334573, 'f1-score': 0.8883459565111166, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0}                | {'precision': 0.8789410348977136, 'recall': 0.9074794997100969, 'f1-score': 0.8929823131469556, 'support': 12073.0} | 0.8963   | {'precision': 0.8631716472110806, 'recall': 0.8405273279652555, 'f1-score': 0.8511290006058785, 'support': 29705.0} | {'precision': 0.8952896579013939, 'recall': 0.8962800875273523, 'f1-score': 0.895480468184721, 'support': 29705.0}  |
| No log        | 7.0   | 287  | 0.2966          | {'precision': 0.6248019914007694, 'recall': 0.6624280230326296, 'f1-score': 0.643065098404565, 'support': 4168.0}   | {'precision': 0.8643994834266036, 'recall': 0.9330855018587361, 'f1-score': 0.8974301675977654, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0}                | {'precision': 0.9038098506950404, 'recall': 0.8724426406029985, 'f1-score': 0.8878492856239727, 'support': 12073.0} | 0.8958   | {'precision': 0.8482528313806034, 'recall': 0.8669227401006067, 'f1-score': 0.8570529828736417, 'support': 29705.0} | {'precision': 0.898436583603221, 'recall': 0.8958424507658643, 'f1-score': 0.8968547139279125, 'support': 29705.0}  |
| No log        | 8.0   | 328  | 0.3421          | {'precision': 0.6529668636013357, 'recall': 0.6098848368522073, 'f1-score': 0.6306909812678327, 'support': 4168.0}  | {'precision': 0.9046478198370868, 'recall': 0.8773234200743495, 'f1-score': 0.8907761264449163, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9999115983026874, 'f1-score': 0.9999557971975424, 'support': 11312.0}                | {'precision': 0.8831158369582729, 'recall': 0.9080593058891742, 'f1-score': 0.8954138930861273, 'support': 12073.0} | 0.8990   | {'precision': 0.8601826300991738, 'recall': 0.8487947902796046, 'f1-score': 0.8542091994991047, 'support': 29705.0} | {'precision': 0.8968936372791452, 'recall': 0.8989732368288167, 'f1-score': 0.8977445596081872, 'support': 29705.0} |
| No log        | 9.0   | 369  | 0.3315          | {'precision': 0.6249177451195438, 'recall': 0.6835412667946257, 'f1-score': 0.6529162369657385, 'support': 4168.0}  | {'precision': 0.9221616261774913, 'recall': 0.8643122676579925, 'f1-score': 0.8923003118253778, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9990275813295615, 'f1-score': 0.9995135541502675, 'support': 11312.0}                | {'precision': 0.9006594521474467, 'recall': 0.8823821751014661, 'f1-score': 0.8914271369398771, 'support': 12073.0} | 0.8976   | {'precision': 0.8619347058611204, 'recall': 0.8573158227209113, 'f1-score': 0.8590393099703152, 'support': 29705.0} | {'precision': 0.901357029017618, 'recall': 0.8975929978118162, 'f1-score': 0.8991847263270282, 'support': 29705.0}  |
| No log        | 10.0  | 410  | 0.3580          | {'precision': 0.6654402102496715, 'recall': 0.6074856046065259, 'f1-score': 0.635143609682679, 'support': 4168.0}   | {'precision': 0.8756989247311828, 'recall': 0.9460966542750929, 'f1-score': 0.9095376368103641, 'support': 2152.0} | {'precision': 0.9999115592111082, 'recall': 0.9994695898161244, 'f1-score': 0.9996905256642645, 'support': 11312.0} | {'precision': 0.8878382784479948, 'recall': 0.9021784146442475, 'f1-score': 0.8949509058789696, 'support': 12073.0} | 0.9011   | {'precision': 0.8572222431599893, 'recall': 0.8638075658354977, 'f1-score': 0.8598306695090694, 'support': 29705.0} | {'precision': 0.898432249649582, 'recall': 0.9010604275374516, 'f1-score': 0.8994393224226316, 'support': 29705.0}  |
| No log        | 11.0  | 451  | 0.3818          | {'precision': 0.6605737496826606, 'recall': 0.6242802303262955, 'f1-score': 0.6419143949673121, 'support': 4168.0}  | {'precision': 0.9070716228467816, 'recall': 0.9298327137546468, 'f1-score': 0.9183111519045434, 'support': 2152.0} | {'precision': 0.9999115826702034, 'recall': 0.9997347949080623, 'f1-score': 0.9998231809742728, 'support': 11312.0} | {'precision': 0.8877551020408163, 'recall': 0.9007703139236313, 'f1-score': 0.8942153517247049, 'support': 12073.0} | 0.9018   | {'precision': 0.8638280143101154, 'recall': 0.8636545132281589, 'f1-score': 0.8635660198927083, 'support': 29705.0} | {'precision': 0.8999884427250537, 'recall': 0.901767379229086, 'f1-score': 0.9007765211808001, 'support': 29705.0}  |
| No log        | 12.0  | 492  | 0.4293          | {'precision': 0.6597707576181158, 'recall': 0.5662188099808061, 'f1-score': 0.6094254357650096, 'support': 4168.0}  | {'precision': 0.9170684667309547, 'recall': 0.8838289962825279, 'f1-score': 0.9001419782300046, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0}                | {'precision': 0.8703326011923439, 'recall': 0.9189927938374887, 'f1-score': 0.8940010475001007, 'support': 12073.0} | 0.8976   | {'precision': 0.8617929563853536, 'recall': 0.8421275474792367, 'f1-score': 0.8508257965126946, 'support': 29705.0} | {'precision': 0.8935526460983838, 'recall': 0.8975929978118162, 'f1-score': 0.8947808316465885, 'support': 29705.0} |
| 0.1673        | 13.0  | 533  | 0.4672          | {'precision': 0.686539643515673, 'recall': 0.5359884836852208, 'f1-score': 0.6019940716787928, 'support': 4168.0}   | {'precision': 0.9140989729225023, 'recall': 0.9098513011152416, 'f1-score': 0.911970190964136, 'support': 2152.0}  | {'precision': 1.0, 'recall': 0.9993811881188119, 'f1-score': 0.9996904982977407, 'support': 11312.0}                | {'precision': 0.8640418332820671, 'recall': 0.9306717468731881, 'f1-score': 0.896119950552299, 'support': 12073.0}  | 0.8999   | {'precision': 0.8661701124300606, 'recall': 0.8439731799481155, 'f1-score': 0.852443677873242, 'support': 29705.0}  | {'precision': 0.8945367876491145, 'recall': 0.8999495034505975, 'f1-score': 0.8954393610999487, 'support': 29705.0} |
| 0.1673        | 14.0  | 574  | 0.4210          | {'precision': 0.6521739130434783, 'recall': 0.6477927063339731, 'f1-score': 0.6499759268175253, 'support': 4168.0}  | {'precision': 0.9036697247706422, 'recall': 0.9154275092936803, 'f1-score': 0.909510618651893, 'support': 2152.0}  | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0}                | {'precision': 0.8944449043795016, 'recall': 0.8948894226787045, 'f1-score': 0.8946671083140114, 'support': 12073.0} | 0.9015   | {'precision': 0.8625721355484055, 'recall': 0.8643948070306205, 'f1-score': 0.8634720945847734, 'support': 29705.0} | {'precision': 0.9013159888182246, 'recall': 0.9015317286652079, 'f1-score': 0.9014200207764028, 'support': 29705.0} |
| 0.1673        | 15.0  | 615  | 0.4440          | {'precision': 0.6588266384778013, 'recall': 0.5981285988483686, 'f1-score': 0.6270120724346077, 'support': 4168.0}  | {'precision': 0.9058606368251039, 'recall': 0.9121747211895911, 'f1-score': 0.9090067145172495, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.999557991513437, 'f1-score': 0.9997789469030461, 'support': 11312.0}                 | {'precision': 0.8805334618783642, 'recall': 0.9078108175267124, 'f1-score': 0.8939641109298531, 'support': 12073.0} | 0.8996   | {'precision': 0.8613051842953173, 'recall': 0.8544180322695273, 'f1-score': 0.857440461196189, 'support': 29705.0}  | {'precision': 0.8967541492974446, 'recall': 0.8996128597879145, 'f1-score': 0.8978925071931304, 'support': 29705.0} |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2