File size: 10,741 Bytes
251f723 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
results:
- task:
name: Token Classification
type: token-classification
dataset:
name: essays_su_g
type: essays_su_g
config: sep_tok
split: train[80%:100%]
args: sep_tok
metrics:
- name: Accuracy
type: accuracy
value: 0.8961117656960108
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# longformer-sep_tok
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3184
- Claim: {'precision': 0.6251433815095205, 'recall': 0.6537907869481766, 'f1-score': 0.6391462413510027, 'support': 4168.0}
- Majorclaim: {'precision': 0.8963831867057673, 'recall': 0.8522304832713755, 'f1-score': 0.873749404478323, 'support': 2152.0}
- O: {'precision': 1.0, 'recall': 0.999557991513437, 'f1-score': 0.9997789469030461, 'support': 11312.0}
- Premise: {'precision': 0.8966063537063287, 'recall': 0.8906651205168558, 'f1-score': 0.8936258622122497, 'support': 12073.0}
- Accuracy: 0.8961
- Macro avg: {'precision': 0.8545332304804041, 'recall': 0.8490610955624612, 'f1-score': 0.8515751137361555, 'support': 29705.0}
- Weighted avg: {'precision': 0.8978738508742299, 'recall': 0.8961117656960108, 'f1-score': 0.8969033743223053, 'support': 29705.0}
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 9
### Training results
| Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log | 1.0 | 41 | 0.3777 | {'precision': 0.44112006137322596, 'recall': 0.2759117082533589, 'f1-score': 0.33948339483394835, 'support': 4168.0} | {'precision': 0.7229235880398671, 'recall': 0.5055762081784386, 'f1-score': 0.5950232430954334, 'support': 2152.0} | {'precision': 0.9991005576542543, 'recall': 0.981966053748232, 'f1-score': 0.9904592064199733, 'support': 11312.0} | {'precision': 0.7974438687392055, 'recall': 0.9561003892984345, 'f1-score': 0.8695946963989755, 'support': 12073.0} | 0.8379 | {'precision': 0.7401470189516381, 'recall': 0.6798885898696161, 'f1-score': 0.6986401351870827, 'support': 29705.0} | {'precision': 0.8188414513630281, 'recall': 0.8378724120518432, 'f1-score': 0.8213481946290806, 'support': 29705.0} |
| No log | 2.0 | 82 | 0.2665 | {'precision': 0.6216867469879518, 'recall': 0.3714011516314779, 'f1-score': 0.4650045058576149, 'support': 4168.0} | {'precision': 0.7490087232355274, 'recall': 0.8777881040892194, 'f1-score': 0.808301240907146, 'support': 2152.0} | {'precision': 0.9999115044247787, 'recall': 0.9988507779349364, 'f1-score': 0.9993808597205024, 'support': 11312.0} | {'precision': 0.8468603001567984, 'recall': 0.9394516690135012, 'f1-score': 0.8907563025210085, 'support': 12073.0} | 0.8779 | {'precision': 0.8043668187012641, 'recall': 0.7968729256672837, 'f1-score': 0.7908607272515679, 'support': 29705.0} | {'precision': 0.8664602079008504, 'recall': 0.8778993435448578, 'f1-score': 0.8664097012738992, 'support': 29705.0} |
| No log | 3.0 | 123 | 0.2423 | {'precision': 0.6248701973001038, 'recall': 0.5774952015355086, 'f1-score': 0.6002493765586034, 'support': 4168.0} | {'precision': 0.8225806451612904, 'recall': 0.8531598513011153, 'f1-score': 0.8375912408759125, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0} | {'precision': 0.886398700771417, 'recall': 0.904166321543941, 'f1-score': 0.895194357880925, 'support': 12073.0} | 0.8909 | {'precision': 0.8334623858082029, 'recall': 0.8335727410491722, 'f1-score': 0.8331924249677761, 'support': 29705.0} | {'precision': 0.8883401462766283, 'recall': 0.8909274532906918, 'f1-score': 0.8894467745743578, 'support': 29705.0} |
| No log | 4.0 | 164 | 0.2482 | {'precision': 0.6370126304228446, 'recall': 0.5566218809980806, 'f1-score': 0.5941101152368758, 'support': 4168.0} | {'precision': 0.85006753714543, 'recall': 0.8773234200743495, 'f1-score': 0.8634804482048937, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0} | {'precision': 0.8778424958110588, 'recall': 0.9112896546011762, 'f1-score': 0.8942534341217588, 'support': 12073.0} | 0.8927 | {'precision': 0.8412306658448333, 'recall': 0.8362424376454172, 'f1-score': 0.8379278443579481, 'support': 29705.0} | {'precision': 0.8885576985512976, 'recall': 0.8927453290691802, 'f1-score': 0.890129015184852, 'support': 29705.0} |
| No log | 5.0 | 205 | 0.2617 | {'precision': 0.6225581395348837, 'recall': 0.642274472168906, 'f1-score': 0.6322626358053851, 'support': 4168.0} | {'precision': 0.8676403468735737, 'recall': 0.883364312267658, 'f1-score': 0.8754317292194335, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0} | {'precision': 0.8983036614040981, 'recall': 0.8860266710842376, 'f1-score': 0.8921229306534341, 'support': 12073.0} | 0.8948 | {'precision': 0.8471255369531389, 'recall': 0.8527837613342315, 'f1-score': 0.8498880050584791, 'support': 29705.0} | {'precision': 0.8961186485839084, 'recall': 0.8948325197778152, 'f1-score': 0.8954317149728881, 'support': 29705.0} |
| No log | 6.0 | 246 | 0.2822 | {'precision': 0.5918902562033488, 'recall': 0.7039347408829175, 'f1-score': 0.6430684931506849, 'support': 4168.0} | {'precision': 0.9098445595854923, 'recall': 0.8159851301115242, 'f1-score': 0.8603625673689368, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9996463932107497, 'f1-score': 0.9998231653404068, 'support': 11312.0} | {'precision': 0.9072111207645526, 'recall': 0.8649051602749938, 'f1-score': 0.8855531526947378, 'support': 12073.0} | 0.8901 | {'precision': 0.8522364841383483, 'recall': 0.8461178561200462, 'f1-score': 0.8472018446386915, 'support': 29705.0} | {'precision': 0.8984933156395887, 'recall': 0.8900858441339842, 'f1-score': 0.8932197469531816, 'support': 29705.0} |
| No log | 7.0 | 287 | 0.3260 | {'precision': 0.6209569633787757, 'recall': 0.557341650671785, 'f1-score': 0.5874320394487292, 'support': 4168.0} | {'precision': 0.9243511871893981, 'recall': 0.7778810408921933, 'f1-score': 0.8448145344436034, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0} | {'precision': 0.8664746184989099, 'recall': 0.9218089952787211, 'f1-score': 0.893285708552394, 'support': 12073.0} | 0.8899 | {'precision': 0.852945692266771, 'recall': 0.8141916204376904, 'f1-score': 0.8313499155782478, 'support': 29705.0} | {'precision': 0.8870661655388541, 'recall': 0.8899175223026426, 'f1-score': 0.8874464157201748, 'support': 29705.0} |
| No log | 8.0 | 328 | 0.3097 | {'precision': 0.6349094330745707, 'recall': 0.647552783109405, 'f1-score': 0.6411687848913172, 'support': 4168.0} | {'precision': 0.896469465648855, 'recall': 0.8731412639405205, 'f1-score': 0.8846516007532956, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9996463932107497, 'f1-score': 0.9998231653404068, 'support': 11312.0} | {'precision': 0.8954356846473029, 'recall': 0.89372981032055, 'f1-score': 0.8945819342536168, 'support': 12073.0} | 0.8980 | {'precision': 0.8567036458426821, 'recall': 0.8535175626453062, 'f1-score': 0.8550563713096592, 'support': 29705.0} | {'precision': 0.8987746112734568, 'recall': 0.8980306345733041, 'f1-score': 0.8983823961899579, 'support': 29705.0} |
| No log | 9.0 | 369 | 0.3184 | {'precision': 0.6251433815095205, 'recall': 0.6537907869481766, 'f1-score': 0.6391462413510027, 'support': 4168.0} | {'precision': 0.8963831867057673, 'recall': 0.8522304832713755, 'f1-score': 0.873749404478323, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.999557991513437, 'f1-score': 0.9997789469030461, 'support': 11312.0} | {'precision': 0.8966063537063287, 'recall': 0.8906651205168558, 'f1-score': 0.8936258622122497, 'support': 12073.0} | 0.8961 | {'precision': 0.8545332304804041, 'recall': 0.8490610955624612, 'f1-score': 0.8515751137361555, 'support': 29705.0} | {'precision': 0.8978738508742299, 'recall': 0.8961117656960108, 'f1-score': 0.8969033743223053, 'support': 29705.0} |
### Framework versions
- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2
|