Hugging Face's logo

Theoreticallyhugo
/
longformer-sep_tok

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
longformer-sep_tok / README.md
Theoreticallyhugo's picture
Theoreticallyhugo
trainer: training complete at 2024-03-04 13:55:05.607845.
70125f9 verified
preview code
|
raw
history blame
42.3 kB
metadata
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
  - generated_from_trainer
datasets:
  - essays_su_g
metrics:
  - accuracy
model-index:
  - name: longformer-sep_tok
    results:
      - task:
          name: Token Classification
          type: token-classification
        dataset:
          name: essays_su_g
          type: essays_su_g
          config: sep_tok
          split: train[0%:20%]
          args: sep_tok
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9126898734177216

longformer-sep_tok

This model is a fine-tuned version of allenai/longformer-base-4096 on the essays_su_g dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7583
  • Claim: {'precision': 0.6883213488718076, 'recall': 0.651337400281558, 'f1-score': 0.6693188667872212, 'support': 4262.0}
  • Majorclaim: {'precision': 0.9794412229836584, 'recall': 0.8581986143187067, 'f1-score': 0.9148202855736091, 'support': 2165.0}
  • O: {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0}
  • Premise: {'precision': 0.8922474318232207, 'recall': 0.9259145640003068, 'f1-score': 0.9087692886714339, 'support': 13039.0}
  • Accuracy: 0.9127
  • Macro avg: {'precision': 0.8898995270338492, 'recall': 0.8588626446501428, 'f1-score': 0.8731756127093493, 'support': 31600.0}
  • Weighted avg: {'precision': 0.9119345620149354, 'recall': 0.9126898734177216, 'f1-score': 0.9118407024834276, 'support': 31600.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss Claim Majorclaim O Premise Accuracy Macro avg Weighted avg
No log 1.0 81 0.3121 {'precision': 0.5093167701863354, 'recall': 0.5194744251525105, 'f1-score': 0.5143454524334997, 'support': 4262.0} {'precision': 0.8568507157464212, 'recall': 0.38706697459584294, 'f1-score': 0.5332484887050588, 'support': 2165.0} {'precision': 0.9965494577719356, 'recall': 0.9996703477830888, 'f1-score': 0.9981074631778162, 'support': 12134.0} {'precision': 0.8576898532227186, 'recall': 0.9276785029526804, 'f1-score': 0.8913123572323336, 'support': 13039.0} 0.8632 {'precision': 0.8051016992318527, 'recall': 0.7084725626210306, 'f1-score': 0.7342534403871771, 'support': 31600.0} {'precision': 0.8639664237626854, 'recall': 0.8632278481012658, 'f1-score': 0.8569456038123432, 'support': 31600.0}
No log 2.0 162 0.2600 {'precision': 0.5738275650215032, 'recall': 0.6574378226184889, 'f1-score': 0.6127938764352105, 'support': 4262.0} {'precision': 0.8416968442834971, 'recall': 0.751501154734411, 'f1-score': 0.7940458760370912, 'support': 2165.0} {'precision': 0.9972866304884065, 'recall': 0.9995879347288611, 'f1-score': 0.9984359565360553, 'support': 12134.0} {'precision': 0.9057201711297734, 'recall': 0.8767543523276324, 'f1-score': 0.8910019095124897, 'support': 13039.0} 0.8858 {'precision': 0.8296328027307951, 'recall': 0.8213203161023483, 'f1-score': 0.8240694046302117, 'support': 31600.0} {'precision': 0.8917306340412297, 'recall': 0.885759493670886, 'f1-score': 0.8880896398015512, 'support': 31600.0}
No log 3.0 243 0.2415 {'precision': 0.6848865810330691, 'recall': 0.5879868606288128, 'f1-score': 0.6327483903547532, 'support': 4262.0} {'precision': 0.9132706374085684, 'recall': 0.8073903002309469, 'f1-score': 0.8570728119637167, 'support': 2165.0} {'precision': 0.9981881073958162, 'recall': 0.9988462172408109, 'f1-score': 0.9985170538803757, 'support': 12134.0} {'precision': 0.8835433921498019, 'recall': 0.9408696985965181, 'f1-score': 0.9113058980834943, 'support': 13039.0} 0.9064 {'precision': 0.8699721794968139, 'recall': 0.8337732691742722, 'f1-score': 0.8499110385705849, 'support': 31600.0} {'precision': 0.9028086811308416, 'recall': 0.9063924050632911, 'f1-score': 0.9035082219774861, 'support': 31600.0}
No log 4.0 324 0.3030 {'precision': 0.5141668000640307, 'recall': 0.7536367902393243, 'f1-score': 0.6112855647540204, 'support': 4262.0} {'precision': 0.8085365853658537, 'recall': 0.9187066974595843, 'f1-score': 0.8601081081081082, 'support': 2165.0} {'precision': 0.9974483496584081, 'recall': 0.9986813911323553, 'f1-score': 0.998064489560598, 'support': 12134.0} {'precision': 0.9381049888309755, 'recall': 0.7729887261293044, 'f1-score': 0.8475802043476433, 'support': 13039.0} 0.8670 {'precision': 0.814564180979817, 'recall': 0.8610034012401421, 'f1-score': 0.8292595916925924, 'support': 31600.0} {'precision': 0.8948370200415249, 'recall': 0.8670253164556962, 'f1-score': 0.8743527193624336, 'support': 31600.0}
No log 5.0 405 0.2891 {'precision': 0.6357466063348416, 'recall': 0.7252463632097607, 'f1-score': 0.6775537045155633, 'support': 4262.0} {'precision': 0.9524782830863566, 'recall': 0.8609699769053117, 'f1-score': 0.9044153323629306, 'support': 2165.0} {'precision': 0.9963039014373717, 'recall': 0.9996703477830888, 'f1-score': 0.9979842856555186, 'support': 12134.0} {'precision': 0.9153577661431065, 'recall': 0.8849605031060664, 'f1-score': 0.8999025151101578, 'support': 13039.0} 0.9058 {'precision': 0.8749716392504191, 'recall': 0.8677117977510569, 'f1-score': 0.8699639594110427, 'support': 31600.0} {'precision': 0.9112711699956358, 'recall': 0.9058227848101266, 'f1-score': 0.9078849145530533, 'support': 31600.0}
No log 6.0 486 0.2946 {'precision': 0.6882758620689655, 'recall': 0.5854059127170342, 'f1-score': 0.6326866996323064, 'support': 4262.0} {'precision': 0.9331645569620253, 'recall': 0.851270207852194, 'f1-score': 0.8903381642512077, 'support': 2165.0} {'precision': 0.997697557766631, 'recall': 0.9999175869457723, 'f1-score': 0.9988063387528299, 'support': 12134.0} {'precision': 0.8788207240407544, 'recall': 0.9327402408160135, 'f1-score': 0.9049780489619764, 'support': 13039.0} 0.9061 {'precision': 0.8744896752095941, 'recall': 0.8423334870827535, 'f1-score': 0.8567023128995801, 'support': 31600.0} {'precision': 0.90249172710976, 'recall': 0.9061075949367089, 'f1-score': 0.9032790422240128, 'support': 31600.0}
0.2207 7.0 567 0.3772 {'precision': 0.686332087560064, 'recall': 0.6032379164711403, 'f1-score': 0.6421078921078921, 'support': 4262.0} {'precision': 0.895910780669145, 'recall': 0.8905311778290993, 'f1-score': 0.8932128793143386, 'support': 2165.0} {'precision': 0.9985987471150676, 'recall': 0.998434151969672, 'f1-score': 0.9985164427594165, 'support': 12134.0} {'precision': 0.8885040530582167, 'recall': 0.9246874760334381, 'f1-score': 0.9062347326092676, 'support': 13039.0} 0.9073 {'precision': 0.8673364171006233, 'recall': 0.8542226805758374, 'f1-score': 0.8600179866977287, 'support': 31600.0} {'precision': 0.9040188526155067, 'recall': 0.9073101265822785, 'f1-score': 0.9051538897093159, 'support': 31600.0}
0.2207 8.0 648 0.4119 {'precision': 0.6727958510136728, 'recall': 0.669638667292351, 'f1-score': 0.6712135465663217, 'support': 4262.0} {'precision': 0.9428426909458776, 'recall': 0.8609699769053117, 'f1-score': 0.9000482858522453, 'support': 2165.0} {'precision': 0.9968773112005916, 'recall': 0.9997527608373167, 'f1-score': 0.9983129654775131, 'support': 12134.0} {'precision': 0.9015289131092946, 'recall': 0.9134902983357619, 'f1-score': 0.9074701916117481, 'support': 13039.0} 0.9101 {'precision': 0.8785111915673591, 'recall': 0.8609629258426853, 'f1-score': 0.8692612473769571, 'support': 31600.0} {'precision': 0.9101219979448788, 'recall': 0.910126582278481, 'f1-score': 0.9099794311982754, 'support': 31600.0}
0.2207 9.0 729 0.4600 {'precision': 0.6591751621872104, 'recall': 0.6675269826372595, 'f1-score': 0.6633247843320123, 'support': 4262.0} {'precision': 0.8955431754874652, 'recall': 0.8909930715935335, 'f1-score': 0.8932623292428803, 'support': 2165.0} {'precision': 0.998025341451374, 'recall': 0.9996703477830888, 'f1-score': 0.9988471673254282, 'support': 12134.0} {'precision': 0.904670160295931, 'recall': 0.9002991026919243, 'f1-score': 0.9024793388429752, 'support': 13039.0} 0.9064 {'precision': 0.8643534598554952, 'recall': 0.8646223761764514, 'f1-score': 0.864478404935824, 'support': 31600.0} {'precision': 0.9067813047291732, 'recall': 0.9064240506329114, 'f1-score': 0.9065962911751319, 'support': 31600.0}
0.2207 10.0 810 0.5107 {'precision': 0.6554770318021201, 'recall': 0.6093383388080713, 'f1-score': 0.6315661478599222, 'support': 4262.0} {'precision': 0.8854118729866544, 'recall': 0.8886836027713626, 'f1-score': 0.8870447210696173, 'support': 2165.0} {'precision': 0.9995880026367832, 'recall': 0.9997527608373167, 'f1-score': 0.9996703749484962, 'support': 12134.0} {'precision': 0.8913646935253957, 'recall': 0.9111895083978833, 'f1-score': 0.9011680825242718, 'support': 13039.0} 0.9029 {'precision': 0.8579604002377383, 'recall': 0.8522410527036585, 'f1-score': 0.8548623316005768, 'support': 31600.0} {'precision': 0.9006982556148451, 'recall': 0.9029430379746836, 'f1-score': 0.9016619525618274, 'support': 31600.0}
0.2207 11.0 891 0.5215 {'precision': 0.6662814883290964, 'recall': 0.6764429845143125, 'f1-score': 0.6713237862382117, 'support': 4262.0} {'precision': 0.9742647058823529, 'recall': 0.8568129330254042, 'f1-score': 0.9117719341361514, 'support': 2165.0} {'precision': 0.9967143091835058, 'recall': 1.0, 'f1-score': 0.9983544512094783, 'support': 12134.0} {'precision': 0.8993558165971959, 'recall': 0.9101158064268732, 'f1-score': 0.9047038194709155, 'support': 13039.0} 0.9095 {'precision': 0.8841540799980377, 'recall': 0.8608429309916474, 'f1-score': 0.8715384977636892, 'support': 31600.0} {'precision': 0.9104369212575127, 'recall': 0.9094620253164557, 'f1-score': 0.909671969221807, 'support': 31600.0}
0.2207 12.0 972 0.4782 {'precision': 0.668459200374094, 'recall': 0.6708118254340685, 'f1-score': 0.6696334465394075, 'support': 4262.0} {'precision': 0.9299649824912456, 'recall': 0.8586605080831409, 'f1-score': 0.8928914505283381, 'support': 2165.0} {'precision': 0.9970419063270337, 'recall': 1.0, 'f1-score': 0.9985187623436471, 'support': 12134.0} {'precision': 0.9022350615782272, 'recall': 0.9101924994248025, 'f1-score': 0.9061963119917535, 'support': 13039.0} 0.9089 {'precision': 0.8744252876926502, 'recall': 0.859916208235503, 'f1-score': 0.8668099928507866, 'support': 31600.0} {'precision': 0.9090093910879326, 'recall': 0.9088607594936708, 'f1-score': 0.9088293706925031, 'support': 31600.0}
0.0294 13.0 1053 0.5245 {'precision': 0.6661948572776598, 'recall': 0.662599718442046, 'f1-score': 0.6643924244206564, 'support': 4262.0} {'precision': 0.8852755194218609, 'recall': 0.9053117782909931, 'f1-score': 0.8951815482986984, 'support': 2165.0} {'precision': 0.997615719805969, 'recall': 1.0, 'f1-score': 0.9988064370086842, 'support': 12134.0} {'precision': 0.9054990757855823, 'recall': 0.9016795766546515, 'f1-score': 0.903585289935826, 'support': 13039.0} 0.9074 {'precision': 0.863646293072768, 'recall': 0.8673977683469227, 'f1-score': 0.8654914249159662, 'support': 31600.0} {'precision': 0.9072093536253022, 'recall': 0.9074367088607594, 'f1-score': 0.9073124831355737, 'support': 31600.0}
0.0294 14.0 1134 0.6250 {'precision': 0.6758620689655173, 'recall': 0.6438291881745659, 'f1-score': 0.6594568613314107, 'support': 4262.0} {'precision': 0.9619395203336809, 'recall': 0.8521939953810623, 'f1-score': 0.9037472446730345, 'support': 2165.0} {'precision': 0.9989294243597134, 'recall': 0.9996703477830888, 'f1-score': 0.9992997487333691, 'support': 12134.0} {'precision': 0.8919059277394465, 'recall': 0.922003221105913, 'f1-score': 0.9067048797043518, 'support': 13039.0} 0.9095 {'precision': 0.8821592353495895, 'recall': 0.8544241881111576, 'f1-score': 0.8673021836105415, 'support': 31600.0} {'precision': 0.9086612096971158, 'recall': 0.9095253164556962, 'f1-score': 0.9087103166236499, 'support': 31600.0}
0.0294 15.0 1215 0.6100 {'precision': 0.6945061043285239, 'recall': 0.5872829657437822, 'f1-score': 0.6364098652428173, 'support': 4262.0} {'precision': 0.8436333471588553, 'recall': 0.9394919168591224, 'f1-score': 0.888986013986014, 'support': 2165.0} {'precision': 0.9981901941428102, 'recall': 1.0, 'f1-score': 0.9990942774804447, 'support': 12134.0} {'precision': 0.8950778166654255, 'recall': 0.9218498351100545, 'f1-score': 0.9082665860661931, 'support': 13039.0} 0.9079 {'precision': 0.8578518655739038, 'recall': 0.8621561794282397, 'f1-score': 0.8581891856938673, 'support': 31600.0} {'precision': 0.9040952746986214, 'recall': 0.9079430379746836, 'f1-score': 0.9051560615382727, 'support': 31600.0}
0.0294 16.0 1296 0.6166 {'precision': 0.689494354442808, 'recall': 0.6590802440168935, 'f1-score': 0.6739443378119001, 'support': 4262.0} {'precision': 0.9622543950361944, 'recall': 0.8595842956120092, 'f1-score': 0.9080263478897291, 'support': 2165.0} {'precision': 0.998929776899646, 'recall': 1.0, 'f1-score': 0.9994646019521437, 'support': 12134.0} {'precision': 0.894905169207884, 'recall': 0.9227701510852059, 'f1-score': 0.9086240749131552, 'support': 13039.0} 0.9125 {'precision': 0.886395923896633, 'recall': 0.8603586726785272, 'f1-score': 0.8725148406417321, 'support': 31600.0} {'precision': 0.9117591176610922, 'recall': 0.9125316455696203, 'f1-score': 0.9118126773359343, 'support': 31600.0}
0.0294 17.0 1377 0.7061 {'precision': 0.7230113636363636, 'recall': 0.5971374941342092, 'f1-score': 0.6540735029555385, 'support': 4262.0} {'precision': 0.9705730511099638, 'recall': 0.8683602771362586, 'f1-score': 0.9166260360799608, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8787489288774636, 'recall': 0.9437840325178312, 'f1-score': 0.9101061272787783, 'support': 13039.0} 0.9134 {'precision': 0.8929803620201252, 'recall': 0.8523204509470748, 'f1-score': 0.8701499190298527, 'support': 31600.0} {'precision': 0.9104358997019689, 'recall': 0.9134493670886076, 'f1-score': 0.9104598400719334, 'support': 31600.0}
0.0294 18.0 1458 0.7096 {'precision': 0.6926689277328464, 'recall': 0.6229469732519943, 'f1-score': 0.6559604694255713, 'support': 4262.0} {'precision': 0.943415122684026, 'recall': 0.8702078521939954, 'f1-score': 0.9053339740509372, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.886598694344605, 'recall': 0.9269882659713168, 'f1-score': 0.9063437312537493, 'support': 13039.0} 0.9101 {'precision': 0.880608891677722, 'recall': 0.8550357728543266, 'f1-score': 0.866878642607207, 'support': 31600.0} {'precision': 0.9077851211298028, 'recall': 0.910126582278481, 'f1-score': 0.9084198630296251, 'support': 31600.0}
0.0084 19.0 1539 0.7004 {'precision': 0.6861152141802068, 'recall': 0.6539183481933365, 'f1-score': 0.669629985583854, 'support': 4262.0} {'precision': 0.9721329046087889, 'recall': 0.8378752886836027, 'f1-score': 0.9000248077400148, 'support': 2165.0} {'precision': 0.9995057253480517, 'recall': 0.9999175869457723, 'f1-score': 0.9997116137271866, 'support': 12134.0} {'precision': 0.8926328234685583, 'recall': 0.9264514149858118, 'f1-score': 0.909227758542827, 'support': 13039.0} 0.9118 {'precision': 0.8875966669014015, 'recall': 0.8545406597021308, 'f1-score': 0.8696485413984707, 'support': 31600.0} {'precision': 0.9112636910725905, 'recall': 0.9118354430379747, 'f1-score': 0.9110265244278837, 'support': 31600.0}
0.0084 20.0 1620 0.7779 {'precision': 0.749671052631579, 'recall': 0.5347254809948381, 'f1-score': 0.624212544508354, 'support': 4262.0} {'precision': 0.8985707699400646, 'recall': 0.900230946882217, 'f1-score': 0.8994000922934934, 'support': 2165.0} {'precision': 0.9986009381943873, 'recall': 1.0, 'f1-score': 0.9992999794111592, 'support': 12134.0} {'precision': 0.8717696629213483, 'recall': 0.9520668762941943, 'f1-score': 0.9101506653469702, 'support': 13039.0} 0.9106 {'precision': 0.8796531059218449, 'recall': 0.8467558260428124, 'f1-score': 0.8582658203899942, 'support': 31600.0} {'precision': 0.905839625383487, 'recall': 0.9106329113924051, 'f1-score': 0.9050808715235495, 'support': 31600.0}
0.0084 21.0 1701 0.7120 {'precision': 0.6549062844542448, 'recall': 0.6968559361801971, 'f1-score': 0.6752301921109469, 'support': 4262.0} {'precision': 0.9001831501831502, 'recall': 0.9080831408775981, 'f1-score': 0.9041158887100482, 'support': 2165.0} {'precision': 0.9979439098610083, 'recall': 1.0, 'f1-score': 0.9989708969662042, 'support': 12134.0} {'precision': 0.9136142116019493, 'recall': 0.8914027149321267, 'f1-score': 0.9023718023368659, 'support': 13039.0} 0.9080 {'precision': 0.8666618890250881, 'recall': 0.8740854479974804, 'f1-score': 0.8701721950310163, 'support': 31600.0} {'precision': 0.910182728222209, 'recall': 0.9080063291139241, 'f1-score': 0.908948758616849, 'support': 31600.0}
0.0084 22.0 1782 0.7197 {'precision': 0.6900751490023322, 'recall': 0.6248240262787423, 'f1-score': 0.655830562738579, 'support': 4262.0} {'precision': 0.8988095238095238, 'recall': 0.9066974595842956, 'f1-score': 0.9027362612094734, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.893724847220152, 'recall': 0.9197024311680344, 'f1-score': 0.9065275730430511, 'support': 13039.0} 0.9099 {'precision': 0.8705494061221795, 'recall': 0.862805979257768, 'f1-score': 0.8662221016990592, 'support': 31600.0} {'precision': 0.907256431850533, 'recall': 0.909873417721519, 'f1-score': 0.9082685887276635, 'support': 31600.0}
0.0084 23.0 1863 0.6413 {'precision': 0.7030443756449949, 'recall': 0.6393711872360395, 'f1-score': 0.669697714426149, 'support': 4262.0} {'precision': 0.9488636363636364, 'recall': 0.8484988452655889, 'f1-score': 0.8958790538892952, 'support': 2165.0} {'precision': 0.9986831275720165, 'recall': 1.0, 'f1-score': 0.999341129962115, 'support': 12134.0} {'precision': 0.8909664173632498, 'recall': 0.9318966178387913, 'f1-score': 0.910971998350639, 'support': 13039.0} 0.9129 {'precision': 0.8853893892359744, 'recall': 0.854941662585105, 'f1-score': 0.8689724741570496, 'support': 31600.0} {'precision': 0.9109492749267248, 'recall': 0.912879746835443, 'f1-score': 0.911329081266103, 'support': 31600.0}
0.0084 24.0 1944 0.6702 {'precision': 0.6949836148222839, 'recall': 0.6468793993430314, 'f1-score': 0.6700692672256654, 'support': 4262.0} {'precision': 0.923004251299008, 'recall': 0.902540415704388, 'f1-score': 0.9126576366184026, 'support': 2165.0} {'precision': 0.9985187623436471, 'recall': 1.0, 'f1-score': 0.9992588322490323, 'support': 12134.0} {'precision': 0.8976354384914697, 'recall': 0.9200092031597515, 'f1-score': 0.9086846191720638, 'support': 13039.0} 0.9127 {'precision': 0.8785355167391021, 'recall': 0.8673572545517927, 'f1-score': 0.8726675888162911, 'support': 31600.0} {'precision': 0.9107790985823738, 'recall': 0.9126898734177216, 'f1-score': 0.9115533044331938, 'support': 31600.0}
0.0042 25.0 2025 0.7151 {'precision': 0.71929298981426, 'recall': 0.5633505396527452, 'f1-score': 0.631842105263158, 'support': 4262.0} {'precision': 0.9381800197823936, 'recall': 0.8762124711316397, 'f1-score': 0.9061380463338906, 'support': 2165.0} {'precision': 0.9994234412321884, 'recall': 1.0, 'f1-score': 0.9997116374871267, 'support': 12134.0} {'precision': 0.8726860060997234, 'recall': 0.9436306465219726, 'f1-score': 0.9067727909204806, 'support': 13039.0} 0.9094 {'precision': 0.8823956142321413, 'recall': 0.8457984143265893, 'f1-score': 0.861116145001164, 'support': 31600.0} {'precision': 0.905150105533637, 'recall': 0.909367088607595, 'f1-score': 0.9053358023109301, 'support': 31600.0}
0.0042 26.0 2106 0.7253 {'precision': 0.6981035366478728, 'recall': 0.6391365556076959, 'f1-score': 0.6673199412052915, 'support': 4262.0} {'precision': 0.9854211663066955, 'recall': 0.8429561200923787, 'f1-score': 0.9086382872790639, 'support': 2165.0} {'precision': 0.998929776899646, 'recall': 1.0, 'f1-score': 0.9994646019521437, 'support': 12134.0} {'precision': 0.8867070589094095, 'recall': 0.9315898458470742, 'f1-score': 0.9085945096865884, 'support': 13039.0} 0.9123 {'precision': 0.892290384690906, 'recall': 0.8534206303867873, 'f1-score': 0.8710043350307719, 'support': 31600.0} {'precision': 0.9111247263375736, 'recall': 0.9123417721518987, 'f1-score': 0.9109489485211033, 'support': 31600.0}
0.0042 27.0 2187 0.7276 {'precision': 0.6392857142857142, 'recall': 0.7139840450492726, 'f1-score': 0.6745732653513632, 'support': 4262.0} {'precision': 0.9270680372001958, 'recall': 0.8748267898383372, 'f1-score': 0.900190114068441, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.9138884499920998, 'recall': 0.8871846000460158, 'f1-score': 0.9003385609215084, 'support': 13039.0} 0.9063 {'precision': 0.86995757648368, 'recall': 0.8689988587334064, 'f1-score': 0.8687239875366115, 'support': 31600.0} {'precision': 0.9106623915743305, 'recall': 0.9062974683544304, 'f1-score': 0.9080682868581241, 'support': 31600.0}
0.0042 28.0 2268 0.7136 {'precision': 0.7208904109589042, 'recall': 0.5926794931956828, 'f1-score': 0.650527942312645, 'support': 4262.0} {'precision': 0.9624936644703497, 'recall': 0.877136258660508, 'f1-score': 0.917834702754954, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8785928785928786, 'recall': 0.9424035585551039, 'f1-score': 0.9093802035152637, 'support': 13039.0} 0.9129 {'precision': 0.8904324439928858, 'recall': 0.8530548276028237, 'f1-score': 0.8694048110703583, 'support': 31600.0} {'precision': 0.9095951582465776, 'recall': 0.912879746835443, 'f1-score': 0.9097965468557895, 'support': 31600.0}
0.0042 29.0 2349 0.7576 {'precision': 0.7076629397327515, 'recall': 0.6088690755513844, 'f1-score': 0.6545592130155128, 'support': 4262.0} {'precision': 0.9607843137254902, 'recall': 0.8600461893764434, 'f1-score': 0.9076285644650256, 'support': 2165.0} {'precision': 0.9997528015820699, 'recall': 0.9999175869457723, 'f1-score': 0.9998351874742482, 'support': 12134.0} {'precision': 0.881737499098059, 'recall': 0.9371884346959123, 'f1-score': 0.9086177410959922, 'support': 13039.0} 0.9117 {'precision': 0.8874843885345927, 'recall': 0.851505321642378, 'f1-score': 0.8676601765126947, 'support': 31600.0} {'precision': 0.9089915580219335, 'recall': 0.9117088607594936, 'f1-score': 0.9093105727500654, 'support': 31600.0}
0.0042 30.0 2430 0.7611 {'precision': 0.6982853982300885, 'recall': 0.5924448615673393, 'f1-score': 0.641025641025641, 'support': 4262.0} {'precision': 0.9578125, 'recall': 0.8494226327944573, 'f1-score': 0.9003671970624235, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8783657643426438, 'recall': 0.938185443668993, 'f1-score': 0.9072906623155084, 'support': 13039.0} 0.9092 {'precision': 0.8835541211305358, 'recall': 0.8450132345076974, 'f1-score': 0.8621399740255358, 'support': 31600.0} {'precision': 0.9061319105238772, 'recall': 0.9092088607594937, 'f1-score': 0.9064559935136012, 'support': 31600.0}
0.0029 31.0 2511 0.7102 {'precision': 0.6699375557537912, 'recall': 0.7048334115438761, 'f1-score': 0.6869426023324948, 'support': 4262.0} {'precision': 0.9437052200614124, 'recall': 0.8517321016166282, 'f1-score': 0.8953629521728576, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.9104798464491363, 'recall': 0.9095022624434389, 'f1-score': 0.9099907918968693, 'support': 13039.0} 0.9127 {'precision': 0.8809688610534376, 'recall': 0.8665169439009858, 'f1-score': 0.8730431855251981, 'support': 31600.0} {'precision': 0.9145931368177233, 'recall': 0.9126898734177216, 'f1-score': 0.9134202621375019, 'support': 31600.0}
0.0029 32.0 2592 0.7759 {'precision': 0.7044058744993325, 'recall': 0.6189582355701548, 'f1-score': 0.6589234419882603, 'support': 4262.0} {'precision': 0.9374045801526718, 'recall': 0.8508083140877598, 'f1-score': 0.8920096852300242, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.888315276666909, 'recall': 0.9369583557021244, 'f1-score': 0.9119886533293521, 'support': 13039.0} 0.9124 {'precision': 0.882469638317081, 'recall': 0.8516812263400098, 'f1-score': 0.8656995440615517, 'support': 31600.0} {'precision': 0.9096646325044515, 'recall': 0.912373417721519, 'f1-score': 0.9102358517229381, 'support': 31600.0}
0.0029 33.0 2673 0.7554 {'precision': 0.6743905382573014, 'recall': 0.6555607695917409, 'f1-score': 0.6648423557406306, 'support': 4262.0} {'precision': 0.93687374749499, 'recall': 0.8637413394919169, 'f1-score': 0.8988223984619081, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8980036025217653, 'recall': 0.9176317202239436, 'f1-score': 0.90771156545158, 'support': 13039.0} 0.9102 {'precision': 0.8772551775558668, 'recall': 0.8592334573269004, 'f1-score': 0.8678131788381723, 'support': 31600.0} {'precision': 0.9095776535504659, 'recall': 0.9102215189873418, 'f1-score': 0.9097360727900697, 'support': 31600.0}
0.0029 34.0 2754 0.7645 {'precision': 0.690516206482593, 'recall': 0.674800563115908, 'f1-score': 0.6825679363949211, 'support': 4262.0} {'precision': 0.9715053763440861, 'recall': 0.8346420323325635, 'f1-score': 0.8978881987577639, 'support': 2165.0} {'precision': 0.9990120204182447, 'recall': 1.0, 'f1-score': 0.999505766062603, 'support': 12134.0} {'precision': 0.9001414848462284, 'recall': 0.9270649589692461, 'f1-score': 0.9134048662535892, 'support': 13039.0} 0.9147 {'precision': 0.8902937720227881, 'recall': 0.8591268886044294, 'f1-score': 0.8733416918672193, 'support': 31600.0} {'precision': 0.9147229711543896, 'recall': 0.9147151898734177, 'f1-score': 0.914269668092085, 'support': 31600.0}
0.0029 35.0 2835 0.7632 {'precision': 0.6734360410831, 'recall': 0.6769122477709996, 'f1-score': 0.6751696700210624, 'support': 4262.0} {'precision': 0.9550102249488752, 'recall': 0.8628175519630485, 'f1-score': 0.9065760737685028, 'support': 2165.0} {'precision': 0.999341129962115, 'recall': 1.0, 'f1-score': 0.9996704564178612, 'support': 12134.0} {'precision': 0.9011196852776517, 'recall': 0.9134902983357619, 'f1-score': 0.9072628251513881, 'support': 13039.0} 0.9113 {'precision': 0.8822267703179354, 'recall': 0.8633050245174525, 'f1-score': 0.8721697563397037, 'support': 31600.0} {'precision': 0.9118191896014586, 'recall': 0.9113291139240506, 'f1-score': 0.9113959376158816, 'support': 31600.0}
0.0029 36.0 2916 0.7550 {'precision': 0.6908957415565345, 'recall': 0.6623650868137025, 'f1-score': 0.6763296597987543, 'support': 4262.0} {'precision': 0.9603612644254892, 'recall': 0.8840646651270208, 'f1-score': 0.9206349206349206, 'support': 2165.0} {'precision': 0.9994234412321884, 'recall': 1.0, 'f1-score': 0.9997116374871267, 'support': 12134.0} {'precision': 0.8958146487294469, 'recall': 0.9192422731804586, 'f1-score': 0.9073772663613309, 'support': 13039.0} 0.9132 {'precision': 0.8866237739859149, 'recall': 0.8664180062802955, 'f1-score': 0.8760133710705331, 'support': 31600.0} {'precision': 0.9123832604015749, 'recall': 0.9131962025316456, 'f1-score': 0.9125786328668064, 'support': 31600.0}
0.0029 37.0 2997 0.7504 {'precision': 0.6987046632124352, 'recall': 0.6328015016424214, 'f1-score': 0.6641221374045803, 'support': 4262.0} {'precision': 0.9660144181256437, 'recall': 0.8665127020785219, 'f1-score': 0.9135622108595083, 'support': 2165.0} {'precision': 0.9986831275720165, 'recall': 1.0, 'f1-score': 0.999341129962115, 'support': 12134.0} {'precision': 0.8879689331770223, 'recall': 0.9294424419050541, 'f1-score': 0.9082324727395361, 'support': 13039.0} 0.9122 {'precision': 0.8878427855217793, 'recall': 0.8571891614064994, 'f1-score': 0.8713144877414349, 'support': 31600.0} {'precision': 0.9103021670730208, 'recall': 0.9122151898734178, 'f1-score': 0.9106582031373505, 'support': 31600.0}
0.0012 38.0 3078 0.7436 {'precision': 0.703168567807351, 'recall': 0.650868137024871, 'f1-score': 0.6760082856098452, 'support': 4262.0} {'precision': 0.9739447628973423, 'recall': 0.8632794457274827, 'f1-score': 0.9152791380999021, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8925656298257225, 'recall': 0.9308996088657105, 'f1-score': 0.9113296794053608, 'support': 13039.0} 0.9150 {'precision': 0.8923579456199566, 'recall': 0.8612617979045161, 'f1-score': 0.8756233747034196, 'support': 31600.0} {'precision': 0.9137550264715008, 'recall': 0.9150316455696202, 'f1-score': 0.9138624848869746, 'support': 31600.0}
0.0012 39.0 3159 0.7662 {'precision': 0.6940740740740741, 'recall': 0.6595495072735805, 'f1-score': 0.6763715110683349, 'support': 4262.0} {'precision': 0.964633521271143, 'recall': 0.869284064665127, 'f1-score': 0.9144800777453839, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8959292824246026, 'recall': 0.9249942480251553, 'f1-score': 0.9102298026489565, 'support': 13039.0} 0.9142 {'precision': 0.8885974249298076, 'recall': 0.8634569549909658, 'f1-score': 0.8752394467903114, 'support': 31600.0} {'precision': 0.913278415579882, 'recall': 0.9141772151898734, 'f1-score': 0.9134028902100695, 'support': 31600.0}
0.0012 40.0 3240 0.7683 {'precision': 0.6839403568809582, 'recall': 0.6564992961051149, 'f1-score': 0.669938944091943, 'support': 4262.0} {'precision': 0.9549138804457954, 'recall': 0.8706697459584296, 'f1-score': 0.9108480309253444, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8959390862944162, 'recall': 0.9204693611473272, 'f1-score': 0.9080385852090032, 'support': 13039.0} 0.9120 {'precision': 0.8835953570194699, 'recall': 0.861909600802718, 'f1-score': 0.8721548925078559, 'support': 31600.0} {'precision': 0.9111865239829873, 'recall': 0.911993670886076, 'f1-score': 0.9113506770312947, 'support': 31600.0}
0.0012 41.0 3321 0.7781 {'precision': 0.7004382572828048, 'recall': 0.6374941342092915, 'f1-score': 0.6674855668836751, 'support': 4262.0} {'precision': 0.9735064935064935, 'recall': 0.8655889145496536, 'f1-score': 0.9163814180929095, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8895072124185399, 'recall': 0.9316665388450035, 'f1-score': 0.9100988912196584, 'support': 13039.0} 0.9137 {'precision': 0.8907600169161369, 'recall': 0.8586873969009872, 'f1-score': 0.873439971500344, 'support': 31600.0} {'precision': 0.9120315194045547, 'recall': 0.9137025316455696, 'f1-score': 0.9122490257537337, 'support': 31600.0}
0.0012 42.0 3402 0.7566 {'precision': 0.7066008316008316, 'recall': 0.6379633974659784, 'f1-score': 0.6705302096177559, 'support': 4262.0} {'precision': 0.9613624809354346, 'recall': 0.8734411085450347, 'f1-score': 0.9152952565343659, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8910463071512309, 'recall': 0.9326635478180842, 'f1-score': 0.9113800726945703, 'support': 13039.0} 0.9147 {'precision': 0.889690610409227, 'recall': 0.8610170134572743, 'f1-score': 0.8742704836363157, 'support': 31600.0} {'precision': 0.912728989113513, 'recall': 0.9147151898734177, 'f1-score': 0.9131455359828712, 'support': 31600.0}
0.0012 43.0 3483 0.7474 {'precision': 0.7157216494845361, 'recall': 0.6515720319099014, 'f1-score': 0.6821419798575289, 'support': 4262.0} {'precision': 0.9691199176531138, 'recall': 0.8697459584295612, 'f1-score': 0.9167478091528724, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8949259422202669, 'recall': 0.936038039726973, 'f1-score': 0.9150204295835364, 'support': 13039.0} 0.9177 {'precision': 0.8948389034536566, 'recall': 0.8643390075166089, 'f1-score': 0.8784260570997677, 'support': 31600.0} {'precision': 0.9160282187313247, 'recall': 0.9176898734177216, 'f1-score': 0.9162816462431637, 'support': 31600.0}
0.0005 44.0 3564 0.7155 {'precision': 0.6937844972402207, 'recall': 0.6783200375410605, 'f1-score': 0.6859651204176058, 'support': 4262.0} {'precision': 0.9562594268476622, 'recall': 0.8785219399538107, 'f1-score': 0.9157438613384691, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.9020668921458098, 'recall': 0.9204693611473272, 'f1-score': 0.9111752201639842, 'support': 13039.0} 0.9155 {'precision': 0.8879247301726005, 'recall': 0.8693278346605496, 'f1-score': 0.8781695529312981, 'support': 31600.0} {'precision': 0.9151349193838588, 'recall': 0.9154746835443038, 'f1-score': 0.9151418675225096, 'support': 31600.0}
0.0005 45.0 3645 0.7882 {'precision': 0.7132675438596491, 'recall': 0.6105114969497888, 'f1-score': 0.6579013906447534, 'support': 4262.0} {'precision': 0.9819148936170212, 'recall': 0.8526558891454965, 'f1-score': 0.9127317676143386, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8815761142611067, 'recall': 0.9420200935654575, 'f1-score': 0.9107963814325967, 'support': 13039.0} 0.9134 {'precision': 0.8940866640486217, 'recall': 0.8512968699151857, 'f1-score': 0.8703058873742054, 'support': 31600.0} {'precision': 0.9110653490487014, 'recall': 0.9134493670886076, 'f1-score': 0.9109941308951929, 'support': 31600.0}
0.0005 46.0 3726 0.7473 {'precision': 0.6906959341481667, 'recall': 0.6496949788831534, 'f1-score': 0.6695683714182081, 'support': 4262.0} {'precision': 0.9745322245322245, 'recall': 0.8660508083140878, 'f1-score': 0.917094644167278, 'support': 2165.0} {'precision': 0.9997528219494108, 'recall': 1.0, 'f1-score': 0.9998763956985703, 'support': 12134.0} {'precision': 0.8926829268292683, 'recall': 0.9262980289899532, 'f1-score': 0.9091798712785577, 'support': 13039.0} 0.9132 {'precision': 0.8894159768647676, 'recall': 0.8605109540467986, 'f1-score': 0.8739298206406536, 'support': 31600.0} {'precision': 0.9121614481617955, 'recall': 0.9131645569620254, 'f1-score': 0.9122312288169027, 'support': 31600.0}
0.0005 47.0 3807 0.7533 {'precision': 0.6924433249370278, 'recall': 0.6450023463162834, 'f1-score': 0.6678814382896016, 'support': 4262.0} {'precision': 0.9795490298898794, 'recall': 0.8628175519630485, 'f1-score': 0.9174852652259332, 'support': 2165.0} {'precision': 0.9996704564178612, 'recall': 1.0, 'f1-score': 0.9998352010547131, 'support': 12134.0} {'precision': 0.8909827015090173, 'recall': 0.9282920469361148, 'f1-score': 0.9092548076923076, 'support': 13039.0} 0.9131 {'precision': 0.8906613781884464, 'recall': 0.8590279863038617, 'f1-score': 0.8736141780656388, 'support': 31600.0} {'precision': 0.912007653915937, 'recall': 0.913132911392405, 'f1-score': 0.912045571401972, 'support': 31600.0}
0.0005 48.0 3888 0.7521 {'precision': 0.6857073407875547, 'recall': 0.6618958235570155, 'f1-score': 0.6735912129894938, 'support': 4262.0} {'precision': 0.98, 'recall': 0.8600461893764434, 'f1-score': 0.9161131611316112, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8951438982672715, 'recall': 0.9231536160748524, 'f1-score': 0.9089330212187571, 'support': 13039.0} 0.9131 {'precision': 0.8901098358778841, 'recall': 0.8612739072520779, 'f1-score': 0.8746078512862488, 'support': 31600.0} {'precision': 0.9128154441588995, 'recall': 0.9131012658227848, 'f1-score': 0.9125730671600638, 'support': 31600.0}
0.0005 49.0 3969 0.7576 {'precision': 0.6885205343889164, 'recall': 0.6529798216799625, 'f1-score': 0.6702793834296725, 'support': 4262.0} {'precision': 0.9794412229836584, 'recall': 0.8581986143187067, 'f1-score': 0.9148202855736091, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8927673421091554, 'recall': 0.9258378710023775, 'f1-score': 0.9090019201084297, 'support': 13039.0} 0.9129 {'precision': 0.89007930098461, 'recall': 0.8592540767502617, 'f1-score': 0.8734738997292112, 'support': 31600.0} {'precision': 0.912175955650765, 'recall': 0.912879746835443, 'f1-score': 0.9120662405605516, 'support': 31600.0}
0.0009 50.0 4050 0.7583 {'precision': 0.6883213488718076, 'recall': 0.651337400281558, 'f1-score': 0.6693188667872212, 'support': 4262.0} {'precision': 0.9794412229836584, 'recall': 0.8581986143187067, 'f1-score': 0.9148202855736091, 'support': 2165.0} {'precision': 0.9995881044567098, 'recall': 1.0, 'f1-score': 0.9997940098051333, 'support': 12134.0} {'precision': 0.8922474318232207, 'recall': 0.9259145640003068, 'f1-score': 0.9087692886714339, 'support': 13039.0} 0.9127 {'precision': 0.8898995270338492, 'recall': 0.8588626446501428, 'f1-score': 0.8731756127093493, 'support': 31600.0} {'precision': 0.9119345620149354, 'recall': 0.9126898734177216, 'f1-score': 0.9118407024834276, 'support': 31600.0}

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2