File size: 9,973 Bytes
251f723 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-sep_tok
results:
- task:
name: Token Classification
type: token-classification
dataset:
name: essays_su_g
type: essays_su_g
config: sep_tok
split: train[80%:100%]
args: sep_tok
metrics:
- name: Accuracy
type: accuracy
value: 0.8973236828816697
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# longformer-sep_tok
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2957
- Claim: {'precision': 0.6376988984088128, 'recall': 0.625, 'f1-score': 0.6312855931176542, 'support': 4168.0}
- Majorclaim: {'precision': 0.9108095007270964, 'recall': 0.8731412639405205, 'f1-score': 0.8915776986951364, 'support': 2152.0}
- O: {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0}
- Premise: {'precision': 0.8869018455005716, 'recall': 0.8996107015654767, 'f1-score': 0.8932110695341092, 'support': 12073.0}
- Accuracy: 0.8973
- Macro avg: {'precision': 0.8588304587809044, 'recall': 0.849393790527843, 'f1-score': 0.8539854382348816, 'support': 29705.0}
- Weighted avg: {'precision': 0.896702809171908, 'recall': 0.8973236828816697, 'f1-score': 0.8969571027061913, 'support': 29705.0}
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 8
### Training results
| Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log | 1.0 | 41 | 0.3532 | {'precision': 0.46718648473034435, 'recall': 0.3450095969289827, 'f1-score': 0.39690863924924097, 'support': 4168.0} | {'precision': 0.72375, 'recall': 0.5381040892193308, 'f1-score': 0.6172707889125799, 'support': 2152.0} | {'precision': 0.999551489056333, 'recall': 0.9850601131541725, 'f1-score': 0.9922528940338379, 'support': 11312.0} | {'precision': 0.8174220044671806, 'recall': 0.9397001573759629, 'f1-score': 0.8743064118372379, 'support': 12073.0} | 0.8444 | {'precision': 0.7519774945634644, 'recall': 0.7019684891696123, 'f1-score': 0.7201846835082242, 'support': 29705.0} | {'precision': 0.8308502128427397, 'recall': 0.8444369634741626, 'f1-score': 0.8336154853914309, 'support': 29705.0} |
| No log | 2.0 | 82 | 0.2649 | {'precision': 0.6305418719211823, 'recall': 0.39923224568138194, 'f1-score': 0.48890847656823855, 'support': 4168.0} | {'precision': 0.7800252844500632, 'recall': 0.8601301115241635, 'f1-score': 0.818121546961326, 'support': 2152.0} | {'precision': 0.9998229775181448, 'recall': 0.9985855728429985, 'f1-score': 0.999203892083149, 'support': 11312.0} | {'precision': 0.8480029861888765, 'recall': 0.9408597697341174, 'f1-score': 0.8920213601382126, 'support': 12073.0} | 0.8810 | {'precision': 0.8145982800195667, 'recall': 0.7997019249456654, 'f1-score': 0.7995638189377315, 'support': 29705.0} | {'precision': 0.8703804244486646, 'recall': 0.8809964652415418, 'f1-score': 0.8709219460558968, 'support': 29705.0} |
| No log | 3.0 | 123 | 0.2493 | {'precision': 0.6082398829839103, 'recall': 0.5986084452975048, 'f1-score': 0.6033857315598549, 'support': 4168.0} | {'precision': 0.7881733495342244, 'recall': 0.904275092936803, 'f1-score': 0.8422419389742479, 'support': 2152.0} | {'precision': 0.9998230088495575, 'recall': 0.9987623762376238, 'f1-score': 0.9992924111091456, 'support': 11312.0} | {'precision': 0.8988507689707622, 'recall': 0.8810569038350037, 'f1-score': 0.8898648931275358, 'support': 12073.0} | 0.8879 | {'precision': 0.8237717525846135, 'recall': 0.8456757045767338, 'f1-score': 0.833696243692696, 'support': 29705.0} | {'precision': 0.8885075270279347, 'recall': 0.8879313246928127, 'f1-score': 0.8878892775966825, 'support': 29705.0} |
| No log | 4.0 | 164 | 0.2464 | {'precision': 0.6338532344668882, 'recall': 0.5947696737044146, 'f1-score': 0.6136898130956802, 'support': 4168.0} | {'precision': 0.8428571428571429, 'recall': 0.9047397769516728, 'f1-score': 0.8727028238458091, 'support': 2152.0} | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8904132095621458, 'recall': 0.897788453574091, 'f1-score': 0.8940856223707004, 'support': 12073.0} | 0.8946 | {'precision': 0.8417587943433282, 'recall': 0.8492802752088883, 'f1-score': 0.8450864127262041, 'support': 29705.0} | {'precision': 0.8926674783141166, 'recall': 0.8946305335802054, 'f1-score': 0.8934762252306426, 'support': 29705.0} |
| No log | 5.0 | 205 | 0.2565 | {'precision': 0.6154180238870792, 'recall': 0.6799424184261037, 'f1-score': 0.6460731790721531, 'support': 4168.0} | {'precision': 0.9039196940726577, 'recall': 0.8787174721189591, 'f1-score': 0.8911404335532517, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9993811881188119, 'f1-score': 0.9996904982977407, 'support': 11312.0} | {'precision': 0.9022472870204221, 'recall': 0.8745962064109998, 'f1-score': 0.8882065948855989, 'support': 12073.0} | 0.8951 | {'precision': 0.8553962512450398, 'recall': 0.8581593212687186, 'f1-score': 0.8562776764521861, 'support': 29705.0} | {'precision': 0.8993478876082566, 'recall': 0.8951018347079617, 'f1-score': 0.8968989853619663, 'support': 29705.0} |
| No log | 6.0 | 246 | 0.2794 | {'precision': 0.6114372689715155, 'recall': 0.6746641074856046, 'f1-score': 0.6414965210448272, 'support': 4168.0} | {'precision': 0.9110212335692619, 'recall': 0.837360594795539, 'f1-score': 0.872639225181598, 'support': 2152.0} | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8996361174578996, 'recall': 0.8805599271100804, 'f1-score': 0.8899958141481791, 'support': 12073.0} | 0.8940 | {'precision': 0.8555015526214533, 'recall': 0.8481019564991497, 'f1-score': 0.8509997379918077, 'support': 29705.0} | {'precision': 0.8982088870349785, 'recall': 0.8939572462548393, 'f1-score': 0.8957110422559299, 'support': 29705.0} |
| No log | 7.0 | 287 | 0.2945 | {'precision': 0.6273562107298212, 'recall': 0.6228406909788867, 'f1-score': 0.6250902961714422, 'support': 4168.0} | {'precision': 0.9130213631739573, 'recall': 0.8341078066914498, 'f1-score': 0.87178241864983, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0} | {'precision': 0.8854539537910836, 'recall': 0.9015157790110163, 'f1-score': 0.8934126821260003, 'support': 12073.0} | 0.8949 | {'precision': 0.8564578819237155, 'recall': 0.8395497678973538, 'f1-score': 0.8475381942038841, 'support': 29705.0} | {'precision': 0.8948570356502946, 'recall': 0.8949335128766202, 'f1-score': 0.8947356751979132, 'support': 29705.0} |
| No log | 8.0 | 328 | 0.2957 | {'precision': 0.6376988984088128, 'recall': 0.625, 'f1-score': 0.6312855931176542, 'support': 4168.0} | {'precision': 0.9108095007270964, 'recall': 0.8731412639405205, 'f1-score': 0.8915776986951364, 'support': 2152.0} | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8869018455005716, 'recall': 0.8996107015654767, 'f1-score': 0.8932110695341092, 'support': 12073.0} | 0.8973 | {'precision': 0.8588304587809044, 'recall': 0.849393790527843, 'f1-score': 0.8539854382348816, 'support': 29705.0} | {'precision': 0.896702809171908, 'recall': 0.8973236828816697, 'f1-score': 0.8969571027061913, 'support': 29705.0} |
### Framework versions
- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2
|