|
--- |
|
library_name: transformers |
|
license: apache-2.0 |
|
base_model: allenai/longformer-base-4096 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- stab-gurevych-essays |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: longformer-sep_tok_full_labels |
|
results: |
|
- task: |
|
name: Token Classification |
|
type: token-classification |
|
dataset: |
|
name: stab-gurevych-essays |
|
type: stab-gurevych-essays |
|
config: sep_tok_full_labels |
|
split: train[0%:20%] |
|
args: sep_tok_full_labels |
|
metrics: |
|
- name: Accuracy |
|
type: accuracy |
|
value: 0.8874031749771744 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# longformer-sep_tok_full_labels |
|
|
|
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the stab-gurevych-essays dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.2775 |
|
- B-claim: {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0} |
|
- B-majorclaim: {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0} |
|
- B-premise: {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0} |
|
- I-claim: {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0} |
|
- I-majorclaim: {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0} |
|
- I-premise: {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0} |
|
- O: {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0} |
|
- Accuracy: 0.8874 |
|
- Macro avg: {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0} |
|
- Weighted avg: {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0} |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 5 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg | |
|
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:| |
|
| No log | 1.0 | 41 | 0.4487 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.7102510460251046, 'recall': 0.9590395480225988, 'f1-score': 0.8161057692307693, 'support': 708.0} | {'precision': 0.5242566510172144, 'recall': 0.0722917565817868, 'f1-score': 0.1270623933244832, 'support': 4634.0} | {'precision': 0.635728952772074, 'recall': 0.6736292428198434, 'f1-score': 0.6541305725755335, 'support': 2298.0} | {'precision': 0.7685153090699018, 'recall': 0.9773712438468886, 'f1-score': 0.8604508262992788, 'support': 13611.0} | {'precision': 0.9707444699912788, 'recall': 0.9973120469169993, 'f1-score': 0.9838489353153878, 'support': 12277.0} | 0.8279 | {'precision': 0.5156423469822248, 'recall': 0.5256634054554452, 'f1-score': 0.49165692810649325, 'support': 33953.0} | {'precision': 0.7884799553707518, 'recall': 0.827879716078108, 'f1-score': 0.7793158674251499, 'support': 33953.0} | |
|
| No log | 2.0 | 82 | 0.3651 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6355475763016158, 'recall': 1.0, 'f1-score': 0.7771679473106476, 'support': 708.0} | {'precision': 0.5581831831831832, 'recall': 0.3208890807078118, 'f1-score': 0.40750890654973965, 'support': 4634.0} | {'precision': 0.888728323699422, 'recall': 0.5352480417754569, 'f1-score': 0.66811515480717, 'support': 2298.0} | {'precision': 0.8056312443233424, 'recall': 0.9775181838219088, 'f1-score': 0.8832901812387971, 'support': 13611.0} | {'precision': 0.9999184139675288, 'recall': 0.9982894844017268, 'f1-score': 0.9991032852368142, 'support': 12277.0} | 0.8537 | {'precision': 0.5554298202107274, 'recall': 0.5474206843867007, 'f1-score': 0.5335979250204527, 'support': 33953.0} | {'precision': 0.8341039518604557, 'recall': 0.8537095396577622, 'f1-score': 0.832398123732452, 'support': 33953.0} | |
|
| No log | 3.0 | 123 | 0.2896 | {'precision': 0.47393364928909953, 'recall': 0.352112676056338, 'f1-score': 0.40404040404040403, 'support': 284.0} | {'precision': 0.9333333333333333, 'recall': 0.2978723404255319, 'f1-score': 0.45161290322580644, 'support': 141.0} | {'precision': 0.7856328392246295, 'recall': 0.9731638418079096, 'f1-score': 0.8694006309148264, 'support': 708.0} | {'precision': 0.6642079381805409, 'recall': 0.4080707811825637, 'f1-score': 0.5055473867130063, 'support': 4634.0} | {'precision': 0.7170077628793226, 'recall': 0.8842471714534378, 'f1-score': 0.7918939984411536, 'support': 2298.0} | {'precision': 0.8606260075228371, 'recall': 0.9413709499669385, 'f1-score': 0.8991894452436927, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9978822187830904, 'f1-score': 0.9989399869536856, 'support': 12277.0} | 0.8782 | {'precision': 0.7763916472042519, 'recall': 0.6935314256679729, 'f1-score': 0.7029463936475108, 'support': 33953.0} | {'precision': 0.8699976208166521, 'recall': 0.8782140017082437, 'f1-score': 0.867649200314498, 'support': 33953.0} | |
|
| No log | 4.0 | 164 | 0.2798 | {'precision': 0.5757575757575758, 'recall': 0.5352112676056338, 'f1-score': 0.5547445255474452, 'support': 284.0} | {'precision': 0.9054054054054054, 'recall': 0.475177304964539, 'f1-score': 0.6232558139534884, 'support': 141.0} | {'precision': 0.8377358490566038, 'recall': 0.940677966101695, 'f1-score': 0.8862275449101796, 'support': 708.0} | {'precision': 0.6079838528818121, 'recall': 0.5850237375917134, 'f1-score': 0.596282854943363, 'support': 4634.0} | {'precision': 0.8411037107516651, 'recall': 0.7693646649260226, 'f1-score': 0.8036363636363636, 'support': 2298.0} | {'precision': 0.8822353864820498, 'recall': 0.9081625156123724, 'f1-score': 0.8950112229382376, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9976378594119084, 'f1-score': 0.9988175331294598, 'support': 12277.0} | 0.8828 | {'precision': 0.8071745400478731, 'recall': 0.7444650451734122, 'f1-score': 0.7654251227226482, 'support': 33953.0} | {'precision': 0.881207953399647, 'recall': 0.882779135864283, 'f1-score': 0.8814327847290655, 'support': 33953.0} | |
|
| No log | 5.0 | 205 | 0.2775 | {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0} | {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0} | {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0} | {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0} | {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0} | {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0} | 0.8874 | {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0} | {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0} | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.45.2 |
|
- Pytorch 2.5.0+cu124 |
|
- Datasets 2.19.1 |
|
- Tokenizers 0.20.1 |
|
|
