metadata

license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-sep_tok
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: sep_tok
          split: test
          args: sep_tok
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8875012488760116

longformer-sep_tok

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

Loss: 0.2595
Claim: {'precision': 0.6180851063829788, 'recall': 0.5465663217309501, 'f1-score': 0.5801298052920619, 'support': 4252.0}
Majorclaim: {'precision': 0.8535564853556485, 'recall': 0.8414298808432631, 'f1-score': 0.8474498038310638, 'support': 2182.0}
O: {'precision': 0.9998243611135506, 'recall': 0.9992978144474678, 'f1-score': 0.9995610184372257, 'support': 11393.0}
Premise: {'precision': 0.8723387540262393, 'recall': 0.9101639344262296, 'f1-score': 0.8908500140398733, 'support': 12200.0}
Accuracy: 0.8875
Macro avg: {'precision': 0.8359511767196043, 'recall': 0.8243644878619776, 'f1-score': 0.8294976604000562, 'support': 30027.0}
Weighted avg: {'precision': 0.8833413217661854, 'recall': 0.8875012488760116, 'f1-score': 0.884944092263729, 'support': 30027.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Claim	Majorclaim	O	Premise	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.3497	{'precision': 0.5091044221479004, 'recall': 0.3222013170272813, 'f1-score': 0.39464208555379515, 'support': 4252.0}	{'precision': 0.6774921064501579, 'recall': 0.6883593033913841, 'f1-score': 0.682882473289384, 'support': 2182.0}	{'precision': 0.9962913907284768, 'recall': 0.9903449486526815, 'f1-score': 0.9933092701822344, 'support': 11393.0}	{'precision': 0.8249963752356096, 'recall': 0.9327868852459016, 'f1-score': 0.875586673847811, 'support': 12200.0}	0.8504	{'precision': 0.7519710736405363, 'recall': 0.7334231135793121, 'f1-score': 0.7366051257183062, 'support': 30027.0}	{'precision': 0.8345390272651644, 'recall': 0.8504013054917241, 'f1-score': 0.8381455903227649, 'support': 30027.0}
No log	2.0	82	0.2760	{'precision': 0.5628667225481978, 'recall': 0.6317027281279398, 'f1-score': 0.5953014184397162, 'support': 4252.0}	{'precision': 0.8442959917780062, 'recall': 0.7529789184234648, 'f1-score': 0.7960271317829457, 'support': 2182.0}	{'precision': 0.9999119795792624, 'recall': 0.9971034845958044, 'f1-score': 0.9985057572294981, 'support': 11393.0}	{'precision': 0.8920321392701708, 'recall': 0.8736065573770492, 'f1-score': 0.8827232068908398, 'support': 12200.0}	0.8774	{'precision': 0.8247767082939093, 'recall': 0.8138479221310645, 'f1-score': 0.8181393785857499, 'support': 30027.0}	{'precision': 0.8828838192552426, 'recall': 0.8774436340626769, 'f1-score': 0.8796533802557692, 'support': 30027.0}
No log	3.0	123	0.2556	{'precision': 0.622185154295246, 'recall': 0.526340545625588, 'f1-score': 0.570263727863422, 'support': 4252.0}	{'precision': 0.8404255319148937, 'recall': 0.8327222731439047, 'f1-score': 0.8365561694290976, 'support': 2182.0}	{'precision': 0.9998242839571253, 'recall': 0.9988589484771351, 'f1-score': 0.9993413830954994, 'support': 11393.0}	{'precision': 0.8682290858295825, 'recall': 0.9170491803278689, 'f1-score': 0.8919716176353345, 'support': 12200.0}	0.8866	{'precision': 0.8326660139992118, 'recall': 0.8187427368936242, 'f1-score': 0.8245332245058383, 'support': 30027.0}	{'precision': 0.8812979219018255, 'recall': 0.8866353615079762, 'f1-score': 0.8831277531997092, 'support': 30027.0}
No log	4.0	164	0.2595	{'precision': 0.6180851063829788, 'recall': 0.5465663217309501, 'f1-score': 0.5801298052920619, 'support': 4252.0}	{'precision': 0.8535564853556485, 'recall': 0.8414298808432631, 'f1-score': 0.8474498038310638, 'support': 2182.0}	{'precision': 0.9998243611135506, 'recall': 0.9992978144474678, 'f1-score': 0.9995610184372257, 'support': 11393.0}	{'precision': 0.8723387540262393, 'recall': 0.9101639344262296, 'f1-score': 0.8908500140398733, 'support': 12200.0}	0.8875	{'precision': 0.8359511767196043, 'recall': 0.8243644878619776, 'f1-score': 0.8294976604000562, 'support': 30027.0}	{'precision': 0.8833413217661854, 'recall': 0.8875012488760116, 'f1-score': 0.884944092263729, 'support': 30027.0}

Framework versions

Transformers 4.37.2
Pytorch 2.2.0+cu121
Datasets 2.17.0
Tokenizers 0.15.2