meta_data/README_s42_e10.md

metadata

license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-sep_tok
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: sep_tok
          split: train[80%:100%]
          args: sep_tok
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8962127587948157

longformer-sep_tok

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

• Loss: 0.3310
• Claim: {'precision': 0.6261209473442171, 'recall': 0.6533109404990403, 'f1-score': 0.6394270282963485, 'support': 4168.0}
• Majorclaim: {'precision': 0.9034416826003824, 'recall': 0.8782527881040892, 'f1-score': 0.8906691800188501, 'support': 2152.0}
• O: {'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0}
• Premise: {'precision': 0.8951990632318502, 'recall': 0.8865236478091609, 'f1-score': 0.8908402347163843, 'support': 12073.0}
• Accuracy: 0.8962
• Macro avg: {'precision': 0.8561462068102073, 'recall': 0.8543892415571037, 'f1-score': 0.8551456934280991, 'support': 29705.0}
• Weighted avg: {'precision': 0.8978830564693184, 'recall': 0.8962127587948157, 'f1-score': 0.8969858736149474, 'support': 29705.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Claim	Majorclaim	O	Premise	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.3618	{'precision': 0.459572685379137, 'recall': 0.2631957773512476, 'f1-score': 0.33470633104500386, 'support': 4168.0}	{'precision': 0.7210526315789474, 'recall': 0.5729553903345725, 'f1-score': 0.6385292594510618, 'support': 2152.0}	{'precision': 0.9996423462088698, 'recall': 0.9883309759547383, 'f1-score': 0.9939544807965861, 'support': 11312.0}	{'precision': 0.8006100942872989, 'recall': 0.956514536569204, 'f1-score': 0.8716458466996264, 'support': 12073.0}	0.8436	{'precision': 0.7452194393635632, 'recall': 0.6952491700524406, 'f1-score': 0.7097089794980695, 'support': 29705.0}	{'precision': 0.8227882209885014, 'recall': 0.8435616899511866, 'f1-score': 0.8259944234340963, 'support': 29705.0}
No log	2.0	82	0.2635	{'precision': 0.6204819277108434, 'recall': 0.42010556621880996, 'f1-score': 0.5010014306151646, 'support': 4168.0}	{'precision': 0.7734307824591573, 'recall': 0.8359665427509294, 'f1-score': 0.8034836980794997, 'support': 2152.0}	{'precision': 0.9996458923512748, 'recall': 0.9982319660537482, 'f1-score': 0.9989384288747346, 'support': 11312.0}	{'precision': 0.8523489932885906, 'recall': 0.9362213203014992, 'f1-score': 0.8923186231941264, 'support': 12073.0}	0.8802	{'precision': 0.8114768989524666, 'recall': 0.7976313488312466, 'f1-score': 0.7989355451908814, 'support': 29705.0}	{'precision': 0.8701900504562087, 'recall': 0.8801548560848342, 'f1-score': 0.8715780214214702, 'support': 29705.0}
No log	3.0	123	0.2378	{'precision': 0.6241271370093908, 'recall': 0.6218809980806143, 'f1-score': 0.623002043023675, 'support': 4168.0}	{'precision': 0.8295557570262919, 'recall': 0.8503717472118959, 'f1-score': 0.8398347865993575, 'support': 2152.0}	{'precision': 0.9996462681287585, 'recall': 0.9992927864214993, 'f1-score': 0.9994694960212203, 'support': 11312.0}	{'precision': 0.8973251370659578, 'recall': 0.8947237637703968, 'f1-score': 0.8960225623159553, 'support': 12073.0}	0.8930	{'precision': 0.8376635748075998, 'recall': 0.8415673238711016, 'f1-score': 0.8395822219900521, 'support': 29705.0}	{'precision': 0.8930473274211594, 'recall': 0.893048308365595, 'f1-score': 0.8930374115440858, 'support': 29705.0}
No log	4.0	164	0.2418	{'precision': 0.6509234828496042, 'recall': 0.591890595009597, 'f1-score': 0.6200050263885398, 'support': 4168.0}	{'precision': 0.8957732949087416, 'recall': 0.866635687732342, 'f1-score': 0.8809636277751535, 'support': 2152.0}	{'precision': 1.0, 'recall': 0.9993811881188119, 'f1-score': 0.9996904982977407, 'support': 11312.0}	{'precision': 0.8806673052362708, 'recall': 0.913857367679947, 'f1-score': 0.896955408316735, 'support': 12073.0}	0.8978	{'precision': 0.8568410207486541, 'recall': 0.8429412096351745, 'f1-score': 0.8494036401945422, 'support': 29705.0}	{'precision': 0.8949688464325286, 'recall': 0.8978286483756943, 'f1-score': 0.8960597959370997, 'support': 29705.0}
No log	5.0	205	0.2518	{'precision': 0.633635729239358, 'recall': 0.6535508637236085, 'f1-score': 0.6434392346758001, 'support': 4168.0}	{'precision': 0.8811835413777162, 'recall': 0.8856877323420075, 'f1-score': 0.8834298957126304, 'support': 2152.0}	{'precision': 1.0, 'recall': 0.9994695898161244, 'f1-score': 0.9997347245556637, 'support': 11312.0}	{'precision': 0.8996397754879786, 'recall': 0.8895055081587012, 'f1-score': 0.8945439400249896, 'support': 12073.0}	0.8980	{'precision': 0.8536147615262633, 'recall': 0.8570534235101104, 'f1-score': 0.855286948742271, 'support': 29705.0}	{'precision': 0.8991971624299228, 'recall': 0.8979969702070358, 'f1-score': 0.8985633414508561, 'support': 29705.0}
No log	6.0	246	0.2668	{'precision': 0.6150532759011562, 'recall': 0.650911708253359, 'f1-score': 0.6324746473948013, 'support': 4168.0}	{'precision': 0.9063266307013241, 'recall': 0.8587360594795539, 'f1-score': 0.8818897637795274, 'support': 2152.0}	{'precision': 1.0, 'recall': 0.9992043847241867, 'f1-score': 0.9996020340481981, 'support': 11312.0}	{'precision': 0.8931559571619813, 'recall': 0.8842044230928519, 'f1-score': 0.8886576482830385, 'support': 12073.0}	0.8934	{'precision': 0.8536339659411154, 'recall': 0.8482641438874878, 'f1-score': 0.8506560233763915, 'support': 29705.0}	{'precision': 0.8957760927130742, 'recall': 0.8934186163945463, 'f1-score': 0.8944703955182431, 'support': 29705.0}
No log	7.0	287	0.3051	{'precision': 0.6341149188353518, 'recall': 0.5904510556621881, 'f1-score': 0.6115045347248106, 'support': 4168.0}	{'precision': 0.9206932773109243, 'recall': 0.8145910780669146, 'f1-score': 0.8643984220907298, 'support': 2152.0}	{'precision': 0.9998231497037757, 'recall': 0.999557991513437, 'f1-score': 0.9996905530259493, 'support': 11312.0}	{'precision': 0.875901990325906, 'recall': 0.9149341505839477, 'f1-score': 0.8949927078269324, 'support': 12073.0}	0.8944	{'precision': 0.8576333340439894, 'recall': 0.8298835689566219, 'f1-score': 0.8426465544171056, 'support': 29705.0}	{'precision': 0.8924116180149008, 'recall': 0.8943612186500589, 'f1-score': 0.8928693082813114, 'support': 29705.0}
No log	8.0	328	0.3102	{'precision': 0.6281418839558375, 'recall': 0.6415547024952015, 'f1-score': 0.6347774480712166, 'support': 4168.0}	{'precision': 0.9114481409001957, 'recall': 0.8657063197026023, 'f1-score': 0.8879885605338417, 'support': 2152.0}	{'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0}	{'precision': 0.890625, 'recall': 0.8923217095999337, 'f1-score': 0.8914725474781745, 'support': 12073.0}	0.8960	{'precision': 0.8575095397301031, 'recall': 0.8497630804034655, 'f1-score': 0.8534712216910115, 'support': 29705.0}	{'precision': 0.8968876178785191, 'recall': 0.8960107725972059, 'f1-score': 0.8963962681095783, 'support': 29705.0}
No log	9.0	369	0.3262	{'precision': 0.6178665496049166, 'recall': 0.675383877159309, 'f1-score': 0.645346171480972, 'support': 4168.0}	{'precision': 0.9207772795216741, 'recall': 0.8587360594795539, 'f1-score': 0.8886751622986293, 'support': 2152.0}	{'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0}	{'precision': 0.898259252999831, 'recall': 0.8804770976559264, 'f1-score': 0.8892792905843476, 'support': 12073.0}	0.8954	{'precision': 0.8591815540477001, 'recall': 0.8535166560277284, 'f1-score': 0.8557367387611907, 'support': 29705.0}	{'precision': 0.8992244988482462, 'recall': 0.8954384783706447, 'f1-score': 0.897037534417961, 'support': 29705.0}
No log	10.0	410	0.3310	{'precision': 0.6261209473442171, 'recall': 0.6533109404990403, 'f1-score': 0.6394270282963485, 'support': 4168.0}	{'precision': 0.9034416826003824, 'recall': 0.8782527881040892, 'f1-score': 0.8906691800188501, 'support': 2152.0}	{'precision': 0.9998231340643792, 'recall': 0.9994695898161244, 'f1-score': 0.9996463306808134, 'support': 11312.0}	{'precision': 0.8951990632318502, 'recall': 0.8865236478091609, 'f1-score': 0.8908402347163843, 'support': 12073.0}	0.8962	{'precision': 0.8561462068102073, 'recall': 0.8543892415571037, 'f1-score': 0.8551456934280991, 'support': 29705.0}	{'precision': 0.8978830564693184, 'recall': 0.8962127587948157, 'f1-score': 0.8969858736149474, 'support': 29705.0}

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.2