metadata

base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-sep_tok
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: sep_tok
          split: train[80%:100%]
          args: sep_tok
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.8985356000673287

longformer-sep_tok

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

Loss: 0.2583
Claim: {'precision': 0.6372016071850626, 'recall': 0.6468330134357005, 'f1-score': 0.6419811882366948, 'support': 4168.0}
Majorclaim: {'precision': 0.892292490118577, 'recall': 0.8392193308550185, 'f1-score': 0.8649425287356322, 'support': 2152.0}
O: {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0}
Premise: {'precision': 0.8961370562556626, 'recall': 0.9011844611944008, 'f1-score': 0.8986536714297514, 'support': 12073.0}
Accuracy: 0.8985
Macro avg: {'precision': 0.8564077883898256, 'recall': 0.8467429000982954, 'f1-score': 0.8513611920675855, 'support': 29705.0}
Weighted avg: {'precision': 0.8990786876841317, 'recall': 0.8985356000673287, 'f1-score': 0.8987402622673225, 'support': 29705.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Claim	Majorclaim	O	Premise	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.3743	{'precision': 0.43195876288659796, 'recall': 0.30158349328214973, 'f1-score': 0.35518508053122355, 'support': 4168.0}	{'precision': 0.6687344913151365, 'recall': 0.5009293680297398, 'f1-score': 0.5727948990435707, 'support': 2152.0}	{'precision': 0.9990991802540312, 'recall': 0.980463224893918, 'f1-score': 0.9896934814616517, 'support': 11312.0}	{'precision': 0.8109643516545946, 'recall': 0.9459123664375052, 'f1-score': 0.8732555916650737, 'support': 12073.0}	0.8364	{'precision': 0.72768919652759, 'recall': 0.6822221131608281, 'f1-score': 0.69773226317538, 'support': 29705.0}	{'precision': 0.8191248373533424, 'recall': 0.836424844302306, 'f1-score': 0.8231372987329588, 'support': 29705.0}
No log	2.0	82	0.2730	{'precision': 0.5999298737727911, 'recall': 0.41050863723608444, 'f1-score': 0.4874643874643875, 'support': 4168.0}	{'precision': 0.749693752552062, 'recall': 0.8531598513011153, 'f1-score': 0.7980873723103673, 'support': 2152.0}	{'precision': 0.9993805309734514, 'recall': 0.9983203677510608, 'f1-score': 0.9988501680523616, 'support': 11312.0}	{'precision': 0.8544719169719169, 'recall': 0.9274413981611861, 'f1-score': 0.8894626047583112, 'support': 12073.0}	0.8765	{'precision': 0.8008690185675553, 'recall': 0.7973575636123617, 'f1-score': 0.7934661331463568, 'support': 29705.0}	{'precision': 0.8663484493974304, 'recall': 0.8765191045278573, 'f1-score': 0.8680932745470084, 'support': 29705.0}
No log	3.0	123	0.2507	{'precision': 0.6004319654427646, 'recall': 0.6002879078694817, 'f1-score': 0.6003599280143971, 'support': 4168.0}	{'precision': 0.7873417721518987, 'recall': 0.8671003717472119, 'f1-score': 0.825298540468819, 'support': 2152.0}	{'precision': 0.9997347480106101, 'recall': 0.999557991513437, 'f1-score': 0.9996463619485456, 'support': 11312.0}	{'precision': 0.8981278461797942, 'recall': 0.8821336867390044, 'f1-score': 0.8900589193932557, 'support': 12073.0}	0.8862	{'precision': 0.8214090829462669, 'recall': 0.8372699894672837, 'f1-score': 0.8288409374562543, 'support': 29705.0}	{'precision': 0.8870243017021042, 'recall': 0.8862144420131292, 'f1-score': 0.8864508877040778, 'support': 29705.0}
No log	4.0	164	0.2533	{'precision': 0.6344515441959532, 'recall': 0.5717370441458733, 'f1-score': 0.6014639071176174, 'support': 4168.0}	{'precision': 0.9013713080168776, 'recall': 0.7941449814126395, 'f1-score': 0.8443675889328064, 'support': 2152.0}	{'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0}	{'precision': 0.8741170930780098, 'recall': 0.9225544603661062, 'f1-score': 0.8976828531130365, 'support': 12073.0}	0.8935	{'precision': 0.8524628839444942, 'recall': 0.8220649206324984, 'f1-score': 0.8358454351890219, 'support': 29705.0}	{'precision': 0.8903673007029912, 'recall': 0.8934522807608147, 'f1-score': 0.8911700264460231, 'support': 29705.0}
No log	5.0	205	0.2560	{'precision': 0.6261571326845479, 'recall': 0.6329174664107485, 'f1-score': 0.6295191504593723, 'support': 4168.0}	{'precision': 0.8881709741550696, 'recall': 0.8303903345724907, 'f1-score': 0.8583093179634967, 'support': 2152.0}	{'precision': 1.0, 'recall': 0.9990275813295615, 'f1-score': 0.9995135541502675, 'support': 11312.0}	{'precision': 0.8916988258477707, 'recall': 0.8995278721113228, 'f1-score': 0.8955962394854031, 'support': 12073.0}	0.8950	{'precision': 0.8515067331718471, 'recall': 0.8404658136060308, 'f1-score': 0.8457345655146349, 'support': 29705.0}	{'precision': 0.8954265877754937, 'recall': 0.8950008416091567, 'f1-score': 0.8951337550993843, 'support': 29705.0}
No log	6.0	246	0.2583	{'precision': 0.6372016071850626, 'recall': 0.6468330134357005, 'f1-score': 0.6419811882366948, 'support': 4168.0}	{'precision': 0.892292490118577, 'recall': 0.8392193308550185, 'f1-score': 0.8649425287356322, 'support': 2152.0}	{'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0}	{'precision': 0.8961370562556626, 'recall': 0.9011844611944008, 'f1-score': 0.8986536714297514, 'support': 12073.0}	0.8985	{'precision': 0.8564077883898256, 'recall': 0.8467429000982954, 'f1-score': 0.8513611920675855, 'support': 29705.0}	{'precision': 0.8990786876841317, 'recall': 0.8985356000673287, 'f1-score': 0.8987402622673225, 'support': 29705.0}

Framework versions

Transformers 4.37.2
Pytorch 2.2.0+cu121
Datasets 2.17.0
Tokenizers 0.15.2