---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: sep_tok
      split: train[80%:100%]
      args: sep_tok
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8962127587948157
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-sep_tok

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3310
- Claim: {'precision': 0.6261209473442171, 'recall': 0.6533109404990403, 'f1-score': 0.6394270282963485, 'support': 4168.0}
- Majorclaim: {'precision': 0.9034416826003824, 'recall': 0.8782527881040892, 'f1-score': 0.8906691800188501, 'support': 2152.0}
- O: {'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0}
- Premise: {'precision': 0.8951990632318502, 'recall': 0.8865236478091609, 'f1-score': 0.8908402347163843, 'support': 12073.0}
- Accuracy: 0.8962
- Macro avg: {'precision': 0.8561462068102073, 'recall': 0.8543892415571037, 'f1-score': 0.8551456934280991, 'support': 29705.0}
- Weighted avg: {'precision': 0.8978830564693184, 'recall': 0.8962127587948157, 'f1-score': 0.8969858736149474, 'support': 29705.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                               | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.3618          | {'precision': 0.459572685379137, 'recall': 0.2631957773512476, 'f1-score': 0.33470633104500386, 'support': 4168.0}  | {'precision': 0.7210526315789474, 'recall': 0.5729553903345725, 'f1-score': 0.6385292594510618, 'support': 2152.0} | {'precision': 0.9996423462088698, 'recall': 0.9883309759547383, 'f1-score': 0.9939544807965861, 'support': 11312.0} | {'precision': 0.8006100942872989, 'recall': 0.956514536569204, 'f1-score': 0.8716458466996264, 'support': 12073.0}  | 0.8436   | {'precision': 0.7452194393635632, 'recall': 0.6952491700524406, 'f1-score': 0.7097089794980695, 'support': 29705.0} | {'precision': 0.8227882209885014, 'recall': 0.8435616899511866, 'f1-score': 0.8259944234340963, 'support': 29705.0} |
| No log        | 2.0   | 82   | 0.2635          | {'precision': 0.6204819277108434, 'recall': 0.42010556621880996, 'f1-score': 0.5010014306151646, 'support': 4168.0} | {'precision': 0.7734307824591573, 'recall': 0.8359665427509294, 'f1-score': 0.8034836980794997, 'support': 2152.0} | {'precision': 0.9996458923512748, 'recall': 0.9982319660537482, 'f1-score': 0.9989384288747346, 'support': 11312.0} | {'precision': 0.8523489932885906, 'recall': 0.9362213203014992, 'f1-score': 0.8923186231941264, 'support': 12073.0} | 0.8802   | {'precision': 0.8114768989524666, 'recall': 0.7976313488312466, 'f1-score': 0.7989355451908814, 'support': 29705.0} | {'precision': 0.8701900504562087, 'recall': 0.8801548560848342, 'f1-score': 0.8715780214214702, 'support': 29705.0} |
| No log        | 3.0   | 123  | 0.2378          | {'precision': 0.6241271370093908, 'recall': 0.6218809980806143, 'f1-score': 0.623002043023675, 'support': 4168.0}   | {'precision': 0.8295557570262919, 'recall': 0.8503717472118959, 'f1-score': 0.8398347865993575, 'support': 2152.0} | {'precision': 0.9996462681287585, 'recall': 0.9992927864214993, 'f1-score': 0.9994694960212203, 'support': 11312.0} | {'precision': 0.8973251370659578, 'recall': 0.8947237637703968, 'f1-score': 0.8960225623159553, 'support': 12073.0} | 0.8930   | {'precision': 0.8376635748075998, 'recall': 0.8415673238711016, 'f1-score': 0.8395822219900521, 'support': 29705.0} | {'precision': 0.8930473274211594, 'recall': 0.893048308365595, 'f1-score': 0.8930374115440858, 'support': 29705.0}  |
| No log        | 4.0   | 164  | 0.2418          | {'precision': 0.6509234828496042, 'recall': 0.591890595009597, 'f1-score': 0.6200050263885398, 'support': 4168.0}   | {'precision': 0.8957732949087416, 'recall': 0.866635687732342, 'f1-score': 0.8809636277751535, 'support': 2152.0}  | {'precision': 1.0, 'recall': 0.9993811881188119, 'f1-score': 0.9996904982977407, 'support': 11312.0}                | {'precision': 0.8806673052362708, 'recall': 0.913857367679947, 'f1-score': 0.896955408316735, 'support': 12073.0}   | 0.8978   | {'precision': 0.8568410207486541, 'recall': 0.8429412096351745, 'f1-score': 0.8494036401945422, 'support': 29705.0} | {'precision': 0.8949688464325286, 'recall': 0.8978286483756943, 'f1-score': 0.8960597959370997, 'support': 29705.0} |
| No log        | 5.0   | 205  | 0.2518          | {'precision': 0.633635729239358, 'recall': 0.6535508637236085, 'f1-score': 0.6434392346758001, 'support': 4168.0}   | {'precision': 0.8811835413777162, 'recall': 0.8856877323420075, 'f1-score': 0.8834298957126304, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0}                | {'precision': 0.8996397754879786, 'recall': 0.8895055081587012, 'f1-score': 0.8945439400249896, 'support': 12073.0} | 0.8980   | {'precision': 0.8536147615262633, 'recall': 0.8570534235101104, 'f1-score': 0.855286948742271, 'support': 29705.0}  | {'precision': 0.8991971624299228, 'recall': 0.8979969702070358, 'f1-score': 0.8985633414508561, 'support': 29705.0} |
| No log        | 6.0   | 246  | 0.2668          | {'precision': 0.6150532759011562, 'recall': 0.650911708253359, 'f1-score': 0.6324746473948013, 'support': 4168.0}   | {'precision': 0.9063266307013241, 'recall': 0.8587360594795539, 'f1-score': 0.8818897637795274, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9992043847241867, 'f1-score': 0.9996020340481981, 'support': 11312.0}                | {'precision': 0.8931559571619813, 'recall': 0.8842044230928519, 'f1-score': 0.8886576482830385, 'support': 12073.0} | 0.8934   | {'precision': 0.8536339659411154, 'recall': 0.8482641438874878, 'f1-score': 0.8506560233763915, 'support': 29705.0} | {'precision': 0.8957760927130742, 'recall': 0.8934186163945463, 'f1-score': 0.8944703955182431, 'support': 29705.0} |
| No log        | 7.0   | 287  | 0.3051          | {'precision': 0.6341149188353518, 'recall': 0.5904510556621881, 'f1-score': 0.6115045347248106, 'support': 4168.0}  | {'precision': 0.9206932773109243, 'recall': 0.8145910780669146, 'f1-score': 0.8643984220907298, 'support': 2152.0} | {'precision': 0.9998231497037757, 'recall': 0.999557991513437, 'f1-score': 0.9996905530259493, 'support': 11312.0}  | {'precision': 0.875901990325906, 'recall': 0.9149341505839477, 'f1-score': 0.8949927078269324, 'support': 12073.0}  | 0.8944   | {'precision': 0.8576333340439894, 'recall': 0.8298835689566219, 'f1-score': 0.8426465544171056, 'support': 29705.0} | {'precision': 0.8924116180149008, 'recall': 0.8943612186500589, 'f1-score': 0.8928693082813114, 'support': 29705.0} |
| No log        | 8.0   | 328  | 0.3102          | {'precision': 0.6281418839558375, 'recall': 0.6415547024952015, 'f1-score': 0.6347774480712166, 'support': 4168.0}  | {'precision': 0.9114481409001957, 'recall': 0.8657063197026023, 'f1-score': 0.8879885605338417, 'support': 2152.0} | {'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0} | {'precision': 0.890625, 'recall': 0.8923217095999337, 'f1-score': 0.8914725474781745, 'support': 12073.0}           | 0.8960   | {'precision': 0.8575095397301031, 'recall': 0.8497630804034655, 'f1-score': 0.8534712216910115, 'support': 29705.0} | {'precision': 0.8968876178785191, 'recall': 0.8960107725972059, 'f1-score': 0.8963962681095783, 'support': 29705.0} |
| No log        | 9.0   | 369  | 0.3262          | {'precision': 0.6178665496049166, 'recall': 0.675383877159309, 'f1-score': 0.645346171480972, 'support': 4168.0}    | {'precision': 0.9207772795216741, 'recall': 0.8587360594795539, 'f1-score': 0.8886751622986293, 'support': 2152.0} | {'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0} | {'precision': 0.898259252999831, 'recall': 0.8804770976559264, 'f1-score': 0.8892792905843476, 'support': 12073.0}  | 0.8954   | {'precision': 0.8591815540477001, 'recall': 0.8535166560277284, 'f1-score': 0.8557367387611907, 'support': 29705.0} | {'precision': 0.8992244988482462, 'recall': 0.8954384783706447, 'f1-score': 0.897037534417961, 'support': 29705.0}  |
| No log        | 10.0  | 410  | 0.3310          | {'precision': 0.6261209473442171, 'recall': 0.6533109404990403, 'f1-score': 0.6394270282963485, 'support': 4168.0}  | {'precision': 0.9034416826003824, 'recall': 0.8782527881040892, 'f1-score': 0.8906691800188501, 'support': 2152.0} | {'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0} | {'precision': 0.8951990632318502, 'recall': 0.8865236478091609, 'f1-score': 0.8908402347163843, 'support': 12073.0} | 0.8962   | {'precision': 0.8561462068102073, 'recall': 0.8543892415571037, 'f1-score': 0.8551456934280991, 'support': 29705.0} | {'precision': 0.8978830564693184, 'recall': 0.8962127587948157, 'f1-score': 0.8969858736149474, 'support': 29705.0} |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2