meta_data/README_s42_e8.md

---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: sep_tok
      split: train[80%:100%]
      args: sep_tok
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8973236828816697
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-sep_tok

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2957
- Claim: {'precision': 0.6376988984088128, 'recall': 0.625, 'f1-score': 0.6312855931176542, 'support': 4168.0}
- Majorclaim: {'precision': 0.9108095007270964, 'recall': 0.8731412639405205, 'f1-score': 0.8915776986951364, 'support': 2152.0}
- O: {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0}
- Premise: {'precision': 0.8869018455005716, 'recall': 0.8996107015654767, 'f1-score': 0.8932110695341092, 'support': 12073.0}
- Accuracy: 0.8973
- Macro avg: {'precision': 0.8588304587809044, 'recall': 0.849393790527843, 'f1-score': 0.8539854382348816, 'support': 29705.0}
- Weighted avg: {'precision': 0.896702809171908, 'recall': 0.8973236828816697, 'f1-score': 0.8969571027061913, 'support': 29705.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 8

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                                | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.3532          | {'precision': 0.46718648473034435, 'recall': 0.3450095969289827, 'f1-score': 0.39690863924924097, 'support': 4168.0} | {'precision': 0.72375, 'recall': 0.5381040892193308, 'f1-score': 0.6172707889125799, 'support': 2152.0}            | {'precision': 0.999551489056333, 'recall': 0.9850601131541725, 'f1-score': 0.9922528940338379, 'support': 11312.0}  | {'precision': 0.8174220044671806, 'recall': 0.9397001573759629, 'f1-score': 0.8743064118372379, 'support': 12073.0} | 0.8444   | {'precision': 0.7519774945634644, 'recall': 0.7019684891696123, 'f1-score': 0.7201846835082242, 'support': 29705.0} | {'precision': 0.8308502128427397, 'recall': 0.8444369634741626, 'f1-score': 0.8336154853914309, 'support': 29705.0} |
| No log        | 2.0   | 82   | 0.2649          | {'precision': 0.6305418719211823, 'recall': 0.39923224568138194, 'f1-score': 0.48890847656823855, 'support': 4168.0} | {'precision': 0.7800252844500632, 'recall': 0.8601301115241635, 'f1-score': 0.818121546961326, 'support': 2152.0}  | {'precision': 0.9998229775181448, 'recall': 0.9985855728429985, 'f1-score': 0.999203892083149, 'support': 11312.0}  | {'precision': 0.8480029861888765, 'recall': 0.9408597697341174, 'f1-score': 0.8920213601382126, 'support': 12073.0} | 0.8810   | {'precision': 0.8145982800195667, 'recall': 0.7997019249456654, 'f1-score': 0.7995638189377315, 'support': 29705.0} | {'precision': 0.8703804244486646, 'recall': 0.8809964652415418, 'f1-score': 0.8709219460558968, 'support': 29705.0} |
| No log        | 3.0   | 123  | 0.2493          | {'precision': 0.6082398829839103, 'recall': 0.5986084452975048, 'f1-score': 0.6033857315598549, 'support': 4168.0}   | {'precision': 0.7881733495342244, 'recall': 0.904275092936803, 'f1-score': 0.8422419389742479, 'support': 2152.0}  | {'precision': 0.9998230088495575, 'recall': 0.9987623762376238, 'f1-score': 0.9992924111091456, 'support': 11312.0} | {'precision': 0.8988507689707622, 'recall': 0.8810569038350037, 'f1-score': 0.8898648931275358, 'support': 12073.0} | 0.8879   | {'precision': 0.8237717525846135, 'recall': 0.8456757045767338, 'f1-score': 0.833696243692696, 'support': 29705.0}  | {'precision': 0.8885075270279347, 'recall': 0.8879313246928127, 'f1-score': 0.8878892775966825, 'support': 29705.0} |
| No log        | 4.0   | 164  | 0.2464          | {'precision': 0.6338532344668882, 'recall': 0.5947696737044146, 'f1-score': 0.6136898130956802, 'support': 4168.0}   | {'precision': 0.8428571428571429, 'recall': 0.9047397769516728, 'f1-score': 0.8727028238458091, 'support': 2152.0} | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8904132095621458, 'recall': 0.897788453574091, 'f1-score': 0.8940856223707004, 'support': 12073.0}  | 0.8946   | {'precision': 0.8417587943433282, 'recall': 0.8492802752088883, 'f1-score': 0.8450864127262041, 'support': 29705.0} | {'precision': 0.8926674783141166, 'recall': 0.8946305335802054, 'f1-score': 0.8934762252306426, 'support': 29705.0} |
| No log        | 5.0   | 205  | 0.2565          | {'precision': 0.6154180238870792, 'recall': 0.6799424184261037, 'f1-score': 0.6460731790721531, 'support': 4168.0}   | {'precision': 0.9039196940726577, 'recall': 0.8787174721189591, 'f1-score': 0.8911404335532517, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9993811881188119, 'f1-score': 0.9996904982977407, 'support': 11312.0}                | {'precision': 0.9022472870204221, 'recall': 0.8745962064109998, 'f1-score': 0.8882065948855989, 'support': 12073.0} | 0.8951   | {'precision': 0.8553962512450398, 'recall': 0.8581593212687186, 'f1-score': 0.8562776764521861, 'support': 29705.0} | {'precision': 0.8993478876082566, 'recall': 0.8951018347079617, 'f1-score': 0.8968989853619663, 'support': 29705.0} |
| No log        | 6.0   | 246  | 0.2794          | {'precision': 0.6114372689715155, 'recall': 0.6746641074856046, 'f1-score': 0.6414965210448272, 'support': 4168.0}   | {'precision': 0.9110212335692619, 'recall': 0.837360594795539, 'f1-score': 0.872639225181598, 'support': 2152.0}   | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8996361174578996, 'recall': 0.8805599271100804, 'f1-score': 0.8899958141481791, 'support': 12073.0} | 0.8940   | {'precision': 0.8555015526214533, 'recall': 0.8481019564991497, 'f1-score': 0.8509997379918077, 'support': 29705.0} | {'precision': 0.8982088870349785, 'recall': 0.8939572462548393, 'f1-score': 0.8957110422559299, 'support': 29705.0} |
| No log        | 7.0   | 287  | 0.2945          | {'precision': 0.6273562107298212, 'recall': 0.6228406909788867, 'f1-score': 0.6250902961714422, 'support': 4168.0}   | {'precision': 0.9130213631739573, 'recall': 0.8341078066914498, 'f1-score': 0.87178241864983, 'support': 2152.0}   | {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0}                | {'precision': 0.8854539537910836, 'recall': 0.9015157790110163, 'f1-score': 0.8934126821260003, 'support': 12073.0} | 0.8949   | {'precision': 0.8564578819237155, 'recall': 0.8395497678973538, 'f1-score': 0.8475381942038841, 'support': 29705.0} | {'precision': 0.8948570356502946, 'recall': 0.8949335128766202, 'f1-score': 0.8947356751979132, 'support': 29705.0} |
| No log        | 8.0   | 328  | 0.2957          | {'precision': 0.6376988984088128, 'recall': 0.625, 'f1-score': 0.6312855931176542, 'support': 4168.0}                | {'precision': 0.9108095007270964, 'recall': 0.8731412639405205, 'f1-score': 0.8915776986951364, 'support': 2152.0} | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8869018455005716, 'recall': 0.8996107015654767, 'f1-score': 0.8932110695341092, 'support': 12073.0} | 0.8973   | {'precision': 0.8588304587809044, 'recall': 0.849393790527843, 'f1-score': 0.8539854382348816, 'support': 29705.0}  | {'precision': 0.896702809171908, 'recall': 0.8973236828816697, 'f1-score': 0.8969571027061913, 'support': 29705.0}  |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2