Hugging Face's logo

Theoreticallyhugo
/
longformer-sep_tok

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Theoreticallyhugo commited on
Commit
01a76e2
•
1 Parent(s): 08dc30c

trainer: training complete at 2024-03-02 14:27:47.907240.

Browse files
Files changed (4) hide show
  1. README.md +28 -28
  2. meta_data/README_s42_e16.md +28 -28
  3. meta_data/meta_s42_e16_cvi1.json +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: sep_tok
20
- split: train[0%:20%]
21
  args: sep_tok
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.9033544303797468
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.4725
36
- - Claim: {'precision': 0.6448837209302326, 'recall': 0.6506335053965274, 'f1-score': 0.6477458537724831, 'support': 4262.0}
37
- - Majorclaim: {'precision': 0.9238191975622143, 'recall': 0.8401847575057737, 'f1-score': 0.8800193517174649, 'support': 2165.0}
38
- - O: {'precision': 0.9978613144690301, 'recall': 0.9997527608373167, 'f1-score': 0.9988061421925816, 'support': 12134.0}
39
- - Premise: {'precision': 0.8974495217853348, 'recall': 0.9067413145179845, 'f1-score': 0.9020714912448022, 'support': 13039.0}
40
- - Accuracy: 0.9034
41
- - Macro avg: {'precision': 0.866003438686703, 'recall': 0.8493280845644006, 'f1-score': 0.857160709731833, 'support': 31600.0}
42
- - Weighted avg: {'precision': 0.9037486229637037, 'recall': 0.9033544303797468, 'f1-score': 0.9034037540807719, 'support': 31600.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.3643 | {'precision': 0.4827682824320538, 'recall': 0.4042702956358517, 'f1-score': 0.44004597114033966, 'support': 4262.0} | {'precision': 0.7040155440414507, 'recall': 0.5020785219399538, 'f1-score': 0.586141817201402, 'support': 2165.0} | {'precision': 0.9970286102385602, 'recall': 0.967858908851162, 'f1-score': 0.9822272404131643, 'support': 12134.0} | {'precision': 0.8283247212401414, 'recall': 0.9343507937725286, 'f1-score': 0.8781489890799006, 'support': 13039.0} | 0.8461 | {'precision': 0.7530342894880515, 'recall': 0.7021396300498741, 'f1-score': 0.7216410044587016, 'support': 31600.0} | {'precision': 0.8379817490335457, 'recall': 0.8461075949367088, 'f1-score': 0.8390190812350419, 'support': 31600.0} |
74
- | No log | 2.0 | 82 | 0.2769 | {'precision': 0.6401985111662531, 'recall': 0.4237447207883623, 'f1-score': 0.5099534095722152, 'support': 4262.0} | {'precision': 0.6977007161703732, 'recall': 0.8549653579676675, 'f1-score': 0.7683686176836863, 'support': 2165.0} | {'precision': 0.9958073002301875, 'recall': 0.9982693258612164, 'f1-score': 0.9970367931516998, 'support': 12134.0} | {'precision': 0.8682137229623264, 'recall': 0.9296725208988419, 'f1-score': 0.8978926706418281, 'support': 13039.0} | 0.8827 | {'precision': 0.800480062632285, 'recall': 0.801662981379022, 'f1-score': 0.7933128727623573, 'support': 31600.0} | {'precision': 0.8747725512594399, 'recall': 0.8826582278481012, 'f1-score': 0.8747660275153002, 'support': 31600.0} |
75
- | No log | 3.0 | 123 | 0.2435 | {'precision': 0.6495901639344263, 'recall': 0.5950258094791178, 'f1-score': 0.621111927504286, 'support': 4262.0} | {'precision': 0.798990748528175, 'recall': 0.8775981524249422, 'f1-score': 0.836451683909311, 'support': 2165.0} | {'precision': 0.9976969896364534, 'recall': 0.9996703477830888, 'f1-score': 0.9986826938909928, 'support': 12134.0} | {'precision': 0.9022796352583586, 'recall': 0.9106526574123782, 'f1-score': 0.9064468109469827, 'support': 13039.0} | 0.9 | {'precision': 0.8371393843393533, 'recall': 0.8457367417748817, 'f1-score': 0.8406732790628931, 'support': 31600.0} | {'precision': 0.8977610027099522, 'recall': 0.9, 'f1-score': 0.8985845793132259, 'support': 31600.0} |
76
- | No log | 4.0 | 164 | 0.2710 | {'precision': 0.5950994175537256, 'recall': 0.6952135147817926, 'f1-score': 0.6412725895465858, 'support': 4262.0} | {'precision': 0.781058282208589, 'recall': 0.9408775981524249, 'f1-score': 0.8535512256442489, 'support': 2165.0} | {'precision': 0.9986824769433466, 'recall': 0.9995055216746332, 'f1-score': 0.9990938298047615, 'support': 12134.0} | {'precision': 0.9320077512848597, 'recall': 0.8483779430937956, 'f1-score': 0.8882286815480969, 'support': 13039.0} | 0.8921 | {'precision': 0.8267119819976302, 'recall': 0.8709936444256616, 'f1-score': 0.8455365816359233, 'support': 31600.0} | {'precision': 0.9018280741401717, 'recall': 0.8920886075949367, 'f1-score': 0.8951158382824038, 'support': 31600.0} |
77
- | No log | 5.0 | 205 | 0.3124 | {'precision': 0.6569706498951782, 'recall': 0.5882214922571563, 'f1-score': 0.6206981926219362, 'support': 4262.0} | {'precision': 0.9552845528455285, 'recall': 0.7598152424942263, 'f1-score': 0.8464111139696424, 'support': 2165.0} | {'precision': 0.9973686374475783, 'recall': 0.9995879347288611, 'f1-score': 0.9984770528915414, 'support': 12134.0} | {'precision': 0.877131141644486, 'recall': 0.9351177237518215, 'f1-score': 0.9051967334818115, 'support': 13039.0} | 0.9011 | {'precision': 0.8716887454581927, 'recall': 0.8206855983080162, 'f1-score': 0.8426957732412328, 'support': 31600.0} | {'precision': 0.8989615180207339, 'recall': 0.9010759493670886, 'f1-score': 0.8986163457707048, 'support': 31600.0} |
78
- | No log | 6.0 | 246 | 0.3000 | {'precision': 0.6428049671292915, 'recall': 0.6194274988268419, 'f1-score': 0.6308997490739635, 'support': 4262.0} | {'precision': 0.9450247388675096, 'recall': 0.7939953810623557, 'f1-score': 0.8629518072289157, 'support': 2165.0} | {'precision': 0.996794608366894, 'recall': 0.9995055216746332, 'f1-score': 0.9981482243529073, 'support': 12134.0} | {'precision': 0.8865773302731916, 'recall': 0.9183986502032364, 'f1-score': 0.9022074888872147, 'support': 13039.0} | 0.9007 | {'precision': 0.8678004111592217, 'recall': 0.8328317629417669, 'f1-score': 0.8485518173857503, 'support': 31600.0} | {'precision': 0.9000253454718111, 'recall': 0.9006962025316456, 'f1-score': 0.8997658036424813, 'support': 31600.0} |
79
- | No log | 7.0 | 287 | 0.3219 | {'precision': 0.5949652067130577, 'recall': 0.6820741435945565, 'f1-score': 0.6355487538259729, 'support': 4262.0} | {'precision': 0.9873817034700315, 'recall': 0.7228637413394919, 'f1-score': 0.8346666666666667, 'support': 2165.0} | {'precision': 0.9974464579901153, 'recall': 0.9979396736443052, 'f1-score': 0.9976930048611683, 'support': 12134.0} | {'precision': 0.8966048194626223, 'recall': 0.8931666538845003, 'f1-score': 0.8948824343015215, 'support': 13039.0} | 0.8933 | {'precision': 0.8690995469089566, 'recall': 0.8240110531157135, 'f1-score': 0.8406977149138323, 'support': 31600.0} | {'precision': 0.900862932318002, 'recall': 0.8932594936708861, 'f1-score': 0.8952576298728666, 'support': 31600.0} |
80
- | No log | 8.0 | 328 | 0.3370 | {'precision': 0.6643180674383493, 'recall': 0.6194274988268419, 'f1-score': 0.6410879067508499, 'support': 4262.0} | {'precision': 0.9216867469879518, 'recall': 0.8480369515011548, 'f1-score': 0.8833293240317538, 'support': 2165.0} | {'precision': 0.9963854431939538, 'recall': 0.9995879347288611, 'f1-score': 0.9979841198008804, 'support': 12134.0} | {'precision': 0.889755590223609, 'recall': 0.918552036199095, 'f1-score': 0.9039245283018866, 'support': 13039.0} | 0.9045 | {'precision': 0.868036461960966, 'recall': 0.8464011053139882, 'f1-score': 0.8565814697213427, 'support': 31600.0} | {'precision': 0.9024822632687415, 'recall': 0.9044936708860759, 'f1-score': 0.9031815151675017, 'support': 31600.0} |
81
- | No log | 9.0 | 369 | 0.3583 | {'precision': 0.6696588868940754, 'recall': 0.6126231816048804, 'f1-score': 0.6398725646366867, 'support': 4262.0} | {'precision': 0.923741738688358, 'recall': 0.8392609699769054, 'f1-score': 0.8794772507260408, 'support': 2165.0} | {'precision': 0.9980212713331684, 'recall': 0.9976100214273941, 'f1-score': 0.9978156040060999, 'support': 12134.0} | {'precision': 0.8865123116501287, 'recall': 0.9249942480251553, 'f1-score': 0.9053445428614323, 'support': 13039.0} | 0.9049 | {'precision': 0.8694835521414326, 'recall': 0.8436221052585838, 'f1-score': 0.8556274905575649, 'support': 31600.0} | {'precision': 0.9026332651318208, 'recall': 0.904873417721519, 'f1-score': 0.9032749098634072, 'support': 31600.0} |
82
- | No log | 10.0 | 410 | 0.4008 | {'precision': 0.6432761242667825, 'recall': 0.6947442515251055, 'f1-score': 0.6680203045685279, 'support': 4262.0} | {'precision': 0.8905405405405405, 'recall': 0.9131639722863741, 'f1-score': 0.9017103762827823, 'support': 2165.0} | {'precision': 0.9983533673637411, 'recall': 0.9993406955661777, 'f1-score': 0.9988467874794069, 'support': 12134.0} | {'precision': 0.9138627187079408, 'recall': 0.8852672750977836, 'f1-score': 0.8993377483443709, 'support': 13039.0} | 0.9053 | {'precision': 0.8615081877197512, 'recall': 0.8731290486188602, 'f1-score': 0.866978804168772, 'support': 31600.0} | {'precision': 0.9082132550860689, 'recall': 0.9052848101265822, 'f1-score': 0.9065119405905274, 'support': 31600.0} |
83
- | No log | 11.0 | 451 | 0.4241 | {'precision': 0.6283344541582605, 'recall': 0.6576724542468325, 'f1-score': 0.6426688066032329, 'support': 4262.0} | {'precision': 0.9258312020460358, 'recall': 0.836027713625866, 'f1-score': 0.8786407766990292, 'support': 2165.0} | {'precision': 0.9981066842278564, 'recall': 0.9992582825119499, 'f1-score': 0.9986821513878593, 'support': 12134.0} | {'precision': 0.8980515495550783, 'recall': 0.897844926758187, 'f1-score': 0.897948226270374, 'support': 13039.0} | 0.9002 | {'precision': 0.8625809724968077, 'recall': 0.8477008442857088, 'f1-score': 0.8544849902401238, 'support': 31600.0} | {'precision': 0.9019970461114446, 'recall': 0.9001582278481013, 'f1-score': 0.900875565904306, 'support': 31600.0} |
84
- | No log | 12.0 | 492 | 0.4143 | {'precision': 0.6269706007669366, 'recall': 0.6905208822149226, 'f1-score': 0.6572130415364003, 'support': 4262.0} | {'precision': 0.9402756508422665, 'recall': 0.8508083140877598, 'f1-score': 0.8933074684772067, 'support': 2165.0} | {'precision': 0.998682042833608, 'recall': 0.9991758694577221, 'f1-score': 0.9989288951141139, 'support': 12134.0} | {'precision': 0.9047395955336925, 'recall': 0.8886417670066723, 'f1-score': 0.8966184322525729, 'support': 13039.0} | 0.9018 | {'precision': 0.8676669724941259, 'recall': 0.8572867081917691, 'f1-score': 0.8615169593450734, 'support': 31600.0} | {'precision': 0.9057833221028166, 'recall': 0.9017721518987342, 'f1-score': 0.9033880887258622, 'support': 31600.0} |
85
- | 0.1652 | 13.0 | 533 | 0.4456 | {'precision': 0.6519438693351737, 'recall': 0.664946034725481, 'f1-score': 0.6583807643164131, 'support': 4262.0} | {'precision': 0.9468421052631579, 'recall': 0.8309468822170901, 'f1-score': 0.8851168511685117, 'support': 2165.0} | {'precision': 0.9975333004440059, 'recall': 0.9998351738915444, 'f1-score': 0.9986829107672046, 'support': 12134.0} | {'precision': 0.8987946327041164, 'recall': 0.909272183449651, 'f1-score': 0.9040030499428136, 'support': 13039.0} | 0.9057 | {'precision': 0.8737784769366135, 'recall': 0.8512500685709417, 'f1-score': 0.8615458940487357, 'support': 31600.0} | {'precision': 0.9067072852030946, 'recall': 0.9057278481012658, 'f1-score': 0.905937057207278, 'support': 31600.0} |
86
- | 0.1652 | 14.0 | 574 | 0.4601 | {'precision': 0.6456975960458324, 'recall': 0.674331299859221, 'f1-score': 0.6597038907379777, 'support': 4262.0} | {'precision': 0.9215880893300248, 'recall': 0.8577367205542725, 'f1-score': 0.8885167464114833, 'support': 2165.0} | {'precision': 0.9972053263192504, 'recall': 0.9998351738915444, 'f1-score': 0.9985185185185186, 'support': 12134.0} | {'precision': 0.9043028994447871, 'recall': 0.8993787867167727, 'f1-score': 0.9018341215826509, 'support': 13039.0} | 0.9047 | {'precision': 0.8671984777849736, 'recall': 0.8578204952554527, 'f1-score': 0.8621433193126575, 'support': 31600.0} | {'precision': 0.9062815285811774, 'recall': 0.904746835443038, 'f1-score': 0.9053903656115826, 'support': 31600.0} |
87
- | 0.1652 | 15.0 | 615 | 0.4733 | {'precision': 0.6527744124061061, 'recall': 0.6320976067573909, 'f1-score': 0.6422696388127309, 'support': 4262.0} | {'precision': 0.9225352112676056, 'recall': 0.8471131639722864, 'f1-score': 0.883216951601252, 'support': 2165.0} | {'precision': 0.9979432332373509, 'recall': 0.9996703477830888, 'f1-score': 0.9988060438881798, 'support': 12134.0} | {'precision': 0.8922730682670668, 'recall': 0.912186517370964, 'f1-score': 0.9021199135348325, 'support': 13039.0} | 0.9035 | {'precision': 0.8663814812945323, 'recall': 0.8477669089709325, 'f1-score': 0.8566031369592488, 'support': 31600.0} | {'precision': 0.9026204116235915, 'recall': 0.9035443037974683, 'f1-score': 0.9029041768973552, 'support': 31600.0} |
88
- | 0.1652 | 16.0 | 656 | 0.4725 | {'precision': 0.6448837209302326, 'recall': 0.6506335053965274, 'f1-score': 0.6477458537724831, 'support': 4262.0} | {'precision': 0.9238191975622143, 'recall': 0.8401847575057737, 'f1-score': 0.8800193517174649, 'support': 2165.0} | {'precision': 0.9978613144690301, 'recall': 0.9997527608373167, 'f1-score': 0.9988061421925816, 'support': 12134.0} | {'precision': 0.8974495217853348, 'recall': 0.9067413145179845, 'f1-score': 0.9020714912448022, 'support': 13039.0} | 0.9034 | {'precision': 0.866003438686703, 'recall': 0.8493280845644006, 'f1-score': 0.857160709731833, 'support': 31600.0} | {'precision': 0.9037486229637037, 'recall': 0.9033544303797468, 'f1-score': 0.9034037540807719, 'support': 31600.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: sep_tok
20
+ split: train[20%:40%]
21
  args: sep_tok
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.9141753288565909
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.4313
36
+ - Claim: {'precision': 0.6927914852443154, 'recall': 0.6577859439595773, 'f1-score': 0.6748350612629594, 'support': 4354.0}
37
+ - Majorclaim: {'precision': 0.9005037783375315, 'recall': 0.913154533844189, 'f1-score': 0.9067850348763474, 'support': 2349.0}
38
+ - O: {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0}
39
+ - Premise: {'precision': 0.9048672566371682, 'recall': 0.9174517720951099, 'f1-score': 0.9111160614836266, 'support': 13374.0}
40
+ - Accuracy: 0.9142
41
+ - Macro avg: {'precision': 0.8745007481053241, 'recall': 0.8720781199093074, 'f1-score': 0.873154126750686, 'support': 32613.0}
42
+ - Weighted avg: {'precision': 0.9127462162898813, 'recall': 0.9141753288565909, 'f1-score': 0.9133792098172753, 'support': 32613.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.3274 | {'precision': 0.5337112171837709, 'recall': 0.4108865411116215, 'f1-score': 0.4643135219309629, 'support': 4354.0} | {'precision': 0.6779310344827586, 'recall': 0.8369518944231588, 'f1-score': 0.7490950657268052, 'support': 2349.0} | {'precision': 0.9961064759634486, 'recall': 1.0, 'f1-score': 0.9980494407069782, 'support': 12536.0} | {'precision': 0.868321718931475, 'recall': 0.8944220128607746, 'f1-score': 0.8811786372007365, 'support': 13374.0} | 0.8663 | {'precision': 0.7690176116403633, 'recall': 0.7855651120988888, 'f1-score': 0.7731591663913707, 'support': 32613.0} | {'precision': 0.8590551035257559, 'recall': 0.8663109802839358, 'f1-score': 0.8609350954068932, 'support': 32613.0} |
74
+ | No log | 2.0 | 82 | 0.2487 | {'precision': 0.6105208603265094, 'recall': 0.5411116214974736, 'f1-score': 0.5737245829782053, 'support': 4354.0} | {'precision': 0.8533737024221453, 'recall': 0.8399318859088974, 'f1-score': 0.8465994421797898, 'support': 2349.0} | {'precision': 0.9999201915403033, 'recall': 0.9994416081684748, 'f1-score': 0.9996808425756004, 'support': 12536.0} | {'precision': 0.8785221391604371, 'recall': 0.9138627187079408, 'f1-score': 0.8958440225756799, 'support': 13374.0} | 0.8917 | {'precision': 0.8355842233623487, 'recall': 0.8235869585706966, 'f1-score': 0.8289622225773189, 'support': 32613.0} | {'precision': 0.8875950468565349, 'recall': 0.8916689663631068, 'f1-score': 0.8892060198210008, 'support': 32613.0} |
75
+ | No log | 3.0 | 123 | 0.2409 | {'precision': 0.648193359375, 'recall': 0.6097841065686724, 'f1-score': 0.6284023668639053, 'support': 4354.0} | {'precision': 0.8872870249017037, 'recall': 0.8646232439335888, 'f1-score': 0.8758085381630012, 'support': 2349.0} | {'precision': 0.9997607464710104, 'recall': 1.0, 'f1-score': 0.9998803589232302, 'support': 12536.0} | {'precision': 0.8920300971583023, 'recall': 0.9130402273067145, 'f1-score': 0.9024128884454791, 'support': 13374.0} | 0.9025 | {'precision': 0.8568178069765041, 'recall': 0.8468618944522439, 'f1-score': 0.8516260380989039, 'support': 32613.0} | {'precision': 0.900545253284536, 'recall': 0.9024928709410358, 'f1-score': 0.9013800727011249, 'support': 32613.0} |
76
+ | No log | 4.0 | 164 | 0.2301 | {'precision': 0.6427889207258835, 'recall': 0.6182820395039045, 'f1-score': 0.6302973542495903, 'support': 4354.0} | {'precision': 0.8526522593320236, 'recall': 0.9237973605789698, 'f1-score': 0.8868001634654679, 'support': 2349.0} | {'precision': 0.9997606510292005, 'recall': 0.9996011486917677, 'f1-score': 0.999680893498205, 'support': 12536.0} | {'precision': 0.9004945301963135, 'recall': 0.8986092418124719, 'f1-score': 0.8995508982035928, 'support': 13374.0} | 0.9018 | {'precision': 0.8489240903208553, 'recall': 0.8600724476467785, 'f1-score': 0.854082327354214, 'support': 32613.0} | {'precision': 0.9008001866175751, 'recall': 0.9018182933186153, 'f1-score': 0.9011744291494633, 'support': 32613.0} |
77
+ | No log | 5.0 | 205 | 0.2631 | {'precision': 0.6787741203178207, 'recall': 0.5493798805695912, 'f1-score': 0.6072607260726073, 'support': 4354.0} | {'precision': 0.9056947608200455, 'recall': 0.8463175819497658, 'f1-score': 0.875, 'support': 2349.0} | {'precision': 0.9997607273887382, 'recall': 0.9999202297383536, 'f1-score': 0.9998404722022812, 'support': 12536.0} | {'precision': 0.8748955140707718, 'recall': 0.9391356363092568, 'f1-score': 0.9058781103498017, 'support': 13374.0} | 0.9038 | {'precision': 0.864781280649344, 'recall': 0.8336883321417418, 'f1-score': 0.8469948271561726, 'support': 32613.0} | {'precision': 0.8989271945775551, 'recall': 0.903780700947475, 'f1-score': 0.899905013603967, 'support': 32613.0} |
78
+ | No log | 6.0 | 246 | 0.2843 | {'precision': 0.7390988372093024, 'recall': 0.467156637574644, 'f1-score': 0.57247396566282, 'support': 4354.0} | {'precision': 0.886744966442953, 'recall': 0.8999574286930608, 'f1-score': 0.8933023452355799, 'support': 2349.0} | {'precision': 0.9996809952946806, 'recall': 0.9999202297383536, 'f1-score': 0.9998005982053838, 'support': 12536.0} | {'precision': 0.8590842147543178, 'recall': 0.9595483774487812, 'f1-score': 0.9065413958745407, 'support': 13374.0} | 0.9050 | {'precision': 0.8711522534253133, 'recall': 0.8316456683637099, 'f1-score': 0.8430295762445812, 'support': 32613.0} | {'precision': 0.8991013862116997, 'recall': 0.9050378683347131, 'f1-score': 0.896835733694634, 'support': 32613.0} |
79
+ | No log | 7.0 | 287 | 0.2983 | {'precision': 0.6895514223194749, 'recall': 0.5790078089113458, 'f1-score': 0.6294631710362047, 'support': 4354.0} | {'precision': 0.8248984115256742, 'recall': 0.9506172839506173, 'f1-score': 0.8833069620253163, 'support': 2349.0} | {'precision': 0.9992028061224489, 'recall': 0.9998404594767071, 'f1-score': 0.9995215311004785, 'support': 12536.0} | {'precision': 0.8964686998394864, 'recall': 0.9187228951697323, 'f1-score': 0.9074593796159526, 'support': 13374.0} | 0.9068 | {'precision': 0.8525303349517711, 'recall': 0.8620471118771007, 'f1-score': 0.854937760944488, 'support': 32613.0} | {'precision': 0.9031788559978264, 'recall': 0.9068469628675682, 'f1-score': 0.9039933265062536, 'support': 32613.0} |
80
+ | No log | 8.0 | 328 | 0.3043 | {'precision': 0.6570592208961945, 'recall': 0.6701883325677538, 'f1-score': 0.6635588402501421, 'support': 4354.0} | {'precision': 0.8852592895059208, 'recall': 0.9229459344401874, 'f1-score': 0.9037098791162985, 'support': 2349.0} | {'precision': 0.9996012441183507, 'recall': 0.9998404594767071, 'f1-score': 0.9997208374875374, 'support': 12536.0} | {'precision': 0.9098907766990292, 'recall': 0.8969642590100194, 'f1-score': 0.9033812787107464, 'support': 13374.0} | 0.9081 | {'precision': 0.8629526328048738, 'recall': 0.8724847463736669, 'f1-score': 0.8675927088911811, 'support': 32613.0} | {'precision': 0.9088458701337473, 'recall': 0.9081041302548064, 'f1-score': 0.9084190763411705, 'support': 32613.0} |
81
+ | No log | 9.0 | 369 | 0.3104 | {'precision': 0.6748837209302325, 'recall': 0.6665135507579237, 'f1-score': 0.6706725213773977, 'support': 4354.0} | {'precision': 0.9106310885218127, 'recall': 0.9152830991911451, 'f1-score': 0.9129511677282377, 'support': 2349.0} | {'precision': 0.9996012123145638, 'recall': 0.9997606892150607, 'f1-score': 0.9996809444045626, 'support': 12536.0} | {'precision': 0.9039063664827792, 'recall': 0.9066098399880365, 'f1-score': 0.9052560848140958, 'support': 13374.0} | 0.9110 | {'precision': 0.8722555970623471, 'recall': 0.8720417947880416, 'f1-score': 0.8721401795810735, 'support': 32613.0} | {'precision': 0.9105988621342419, 'recall': 0.910986416459694, 'f1-score': 0.9107878958829343, 'support': 32613.0} |
82
+ | No log | 10.0 | 410 | 0.3317 | {'precision': 0.657001414427157, 'recall': 0.6401010564997703, 'f1-score': 0.6484411354118194, 'support': 4354.0} | {'precision': 0.8625536899648575, 'recall': 0.9404001702852277, 'f1-score': 0.8997963340122199, 'support': 2349.0} | {'precision': 0.999680969851651, 'recall': 0.9998404594767071, 'f1-score': 0.9997607083034219, 'support': 12536.0} | {'precision': 0.905666063893912, 'recall': 0.898758785703604, 'f1-score': 0.9021992043833972, 'support': 13374.0} | 0.9061 | {'precision': 0.8562255345343944, 'recall': 0.8697751179913272, 'f1-score': 0.8625493455277146, 'support': 32613.0} | {'precision': 0.905500915362609, 'recall': 0.9060803973875449, 'f1-score': 0.9056494861218845, 'support': 32613.0} |
83
+ | No log | 11.0 | 451 | 0.3783 | {'precision': 0.6712632356562424, 'recall': 0.6260909508497933, 'f1-score': 0.6478906714200832, 'support': 4354.0} | {'precision': 0.8602825745682888, 'recall': 0.9331630481055768, 'f1-score': 0.8952419848887073, 'support': 2349.0} | {'precision': 0.999680969851651, 'recall': 0.9998404594767071, 'f1-score': 0.9997607083034219, 'support': 12536.0} | {'precision': 0.9027922174365067, 'recall': 0.9090025422461493, 'f1-score': 0.9058867362146051, 'support': 13374.0} | 0.9079 | {'precision': 0.8585047493781722, 'recall': 0.8670242501695566, 'f1-score': 0.8621950252067043, 'support': 32613.0} | {'precision': 0.9060628476302188, 'recall': 0.9078894919203998, 'f1-score': 0.9067601525554976, 'support': 32613.0} |
84
+ | No log | 12.0 | 492 | 0.4069 | {'precision': 0.6924342105263158, 'recall': 0.5801561782269178, 'f1-score': 0.6313421644588852, 'support': 4354.0} | {'precision': 0.8779304769603881, 'recall': 0.9246487867177522, 'f1-score': 0.9006842214389384, 'support': 2349.0} | {'precision': 0.9996809952946806, 'recall': 0.9999202297383536, 'f1-score': 0.9998005982053838, 'support': 12536.0} | {'precision': 0.8889048165137615, 'recall': 0.927321668909825, 'f1-score': 0.9077069457659372, 'support': 13374.0} | 0.9087 | {'precision': 0.8647376248237866, 'recall': 0.8580117158982121, 'f1-score': 0.8598834824672862, 'support': 32613.0} | {'precision': 0.9044654345224509, 'recall': 0.908686720019624, 'f1-score': 0.905704596694275, 'support': 32613.0} |
85
+ | 0.1707 | 13.0 | 533 | 0.3896 | {'precision': 0.6659044134461468, 'recall': 0.6688102893890675, 'f1-score': 0.667354188151713, 'support': 4354.0} | {'precision': 0.8783068783068783, 'recall': 0.918688803746275, 'f1-score': 0.8980441115272575, 'support': 2349.0} | {'precision': 0.9998404849258254, 'recall': 1.0, 'f1-score': 0.9999202361011406, 'support': 12536.0} | {'precision': 0.9120422801057002, 'recall': 0.9032451024375654, 'f1-score': 0.9076223749953041, 'support': 13374.0} | 0.9103 | {'precision': 0.8640235141961377, 'recall': 0.8726860488932269, 'f1-score': 0.8682352276938539, 'support': 32613.0} | {'precision': 0.9105002436590061, 'recall': 0.9102505135988717, 'f1-score': 0.9103335319087843, 'support': 32613.0} |
86
+ | 0.1707 | 14.0 | 574 | 0.4518 | {'precision': 0.6987791932059448, 'recall': 0.6047312815801562, 'f1-score': 0.6483624722974637, 'support': 4354.0} | {'precision': 0.8978132884777124, 'recall': 0.9088974031502767, 'f1-score': 0.9033213454622383, 'support': 2349.0} | {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0} | {'precision': 0.8916008614501076, 'recall': 0.9286675639300135, 'f1-score': 0.9097568121886903, 'support': 13374.0} | 0.9114 | {'precision': 0.8720084538340115, 'recall': 0.8605541195997, 'f1-score': 0.8653302448320508, 'support': 32613.0} | {'precision': 0.9079115108212787, 'recall': 0.9113850305093061, 'f1-score': 0.9090381047714349, 'support': 32613.0} |
87
+ | 0.1707 | 15.0 | 615 | 0.4267 | {'precision': 0.6925925925925925, 'recall': 0.6442351860358291, 'f1-score': 0.6675392670157068, 'support': 4354.0} | {'precision': 0.904277848369335, 'recall': 0.9088974031502767, 'f1-score': 0.9065817409766455, 'support': 2349.0} | {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0} | {'precision': 0.9008415660446396, 'recall': 0.9204426499177508, 'f1-score': 0.9105366322719035, 'support': 13374.0} | 0.9133 | {'precision': 0.8743881198022121, 'recall': 0.8683738672105525, 'f1-score': 0.8711344974110167, 'support': 32613.0} | {'precision': 0.911340633421535, 'recall': 0.9132861128997639, 'f1-score': 0.9121529285245232, 'support': 32613.0} |
88
+ | 0.1707 | 16.0 | 656 | 0.4313 | {'precision': 0.6927914852443154, 'recall': 0.6577859439595773, 'f1-score': 0.6748350612629594, 'support': 4354.0} | {'precision': 0.9005037783375315, 'recall': 0.913154533844189, 'f1-score': 0.9067850348763474, 'support': 2349.0} | {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0} | {'precision': 0.9048672566371682, 'recall': 0.9174517720951099, 'f1-score': 0.9111160614836266, 'support': 13374.0} | 0.9142 | {'precision': 0.8745007481053241, 'recall': 0.8720781199093074, 'f1-score': 0.873154126750686, 'support': 32613.0} | {'precision': 0.9127462162898813, 'recall': 0.9141753288565909, 'f1-score': 0.9133792098172753, 'support': 32613.0} |
89
 
90
 
91
  ### Framework versions
meta_data/README_s42_e16.md CHANGED
@@ -17,12 +17,12 @@ model-index:
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: sep_tok
20
- split: train[0%:20%]
21
  args: sep_tok
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.9033544303797468
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.4725
36
- - Claim: {'precision': 0.6448837209302326, 'recall': 0.6506335053965274, 'f1-score': 0.6477458537724831, 'support': 4262.0}
37
- - Majorclaim: {'precision': 0.9238191975622143, 'recall': 0.8401847575057737, 'f1-score': 0.8800193517174649, 'support': 2165.0}
38
- - O: {'precision': 0.9978613144690301, 'recall': 0.9997527608373167, 'f1-score': 0.9988061421925816, 'support': 12134.0}
39
- - Premise: {'precision': 0.8974495217853348, 'recall': 0.9067413145179845, 'f1-score': 0.9020714912448022, 'support': 13039.0}
40
- - Accuracy: 0.9034
41
- - Macro avg: {'precision': 0.866003438686703, 'recall': 0.8493280845644006, 'f1-score': 0.857160709731833, 'support': 31600.0}
42
- - Weighted avg: {'precision': 0.9037486229637037, 'recall': 0.9033544303797468, 'f1-score': 0.9034037540807719, 'support': 31600.0}
43
 
44
  ## Model description
45
 
@@ -68,24 +68,24 @@ The following hyperparameters were used during training:
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 41 | 0.3643 | {'precision': 0.4827682824320538, 'recall': 0.4042702956358517, 'f1-score': 0.44004597114033966, 'support': 4262.0} | {'precision': 0.7040155440414507, 'recall': 0.5020785219399538, 'f1-score': 0.586141817201402, 'support': 2165.0} | {'precision': 0.9970286102385602, 'recall': 0.967858908851162, 'f1-score': 0.9822272404131643, 'support': 12134.0} | {'precision': 0.8283247212401414, 'recall': 0.9343507937725286, 'f1-score': 0.8781489890799006, 'support': 13039.0} | 0.8461 | {'precision': 0.7530342894880515, 'recall': 0.7021396300498741, 'f1-score': 0.7216410044587016, 'support': 31600.0} | {'precision': 0.8379817490335457, 'recall': 0.8461075949367088, 'f1-score': 0.8390190812350419, 'support': 31600.0} |
74
- | No log | 2.0 | 82 | 0.2769 | {'precision': 0.6401985111662531, 'recall': 0.4237447207883623, 'f1-score': 0.5099534095722152, 'support': 4262.0} | {'precision': 0.6977007161703732, 'recall': 0.8549653579676675, 'f1-score': 0.7683686176836863, 'support': 2165.0} | {'precision': 0.9958073002301875, 'recall': 0.9982693258612164, 'f1-score': 0.9970367931516998, 'support': 12134.0} | {'precision': 0.8682137229623264, 'recall': 0.9296725208988419, 'f1-score': 0.8978926706418281, 'support': 13039.0} | 0.8827 | {'precision': 0.800480062632285, 'recall': 0.801662981379022, 'f1-score': 0.7933128727623573, 'support': 31600.0} | {'precision': 0.8747725512594399, 'recall': 0.8826582278481012, 'f1-score': 0.8747660275153002, 'support': 31600.0} |
75
- | No log | 3.0 | 123 | 0.2435 | {'precision': 0.6495901639344263, 'recall': 0.5950258094791178, 'f1-score': 0.621111927504286, 'support': 4262.0} | {'precision': 0.798990748528175, 'recall': 0.8775981524249422, 'f1-score': 0.836451683909311, 'support': 2165.0} | {'precision': 0.9976969896364534, 'recall': 0.9996703477830888, 'f1-score': 0.9986826938909928, 'support': 12134.0} | {'precision': 0.9022796352583586, 'recall': 0.9106526574123782, 'f1-score': 0.9064468109469827, 'support': 13039.0} | 0.9 | {'precision': 0.8371393843393533, 'recall': 0.8457367417748817, 'f1-score': 0.8406732790628931, 'support': 31600.0} | {'precision': 0.8977610027099522, 'recall': 0.9, 'f1-score': 0.8985845793132259, 'support': 31600.0} |
76
- | No log | 4.0 | 164 | 0.2710 | {'precision': 0.5950994175537256, 'recall': 0.6952135147817926, 'f1-score': 0.6412725895465858, 'support': 4262.0} | {'precision': 0.781058282208589, 'recall': 0.9408775981524249, 'f1-score': 0.8535512256442489, 'support': 2165.0} | {'precision': 0.9986824769433466, 'recall': 0.9995055216746332, 'f1-score': 0.9990938298047615, 'support': 12134.0} | {'precision': 0.9320077512848597, 'recall': 0.8483779430937956, 'f1-score': 0.8882286815480969, 'support': 13039.0} | 0.8921 | {'precision': 0.8267119819976302, 'recall': 0.8709936444256616, 'f1-score': 0.8455365816359233, 'support': 31600.0} | {'precision': 0.9018280741401717, 'recall': 0.8920886075949367, 'f1-score': 0.8951158382824038, 'support': 31600.0} |
77
- | No log | 5.0 | 205 | 0.3124 | {'precision': 0.6569706498951782, 'recall': 0.5882214922571563, 'f1-score': 0.6206981926219362, 'support': 4262.0} | {'precision': 0.9552845528455285, 'recall': 0.7598152424942263, 'f1-score': 0.8464111139696424, 'support': 2165.0} | {'precision': 0.9973686374475783, 'recall': 0.9995879347288611, 'f1-score': 0.9984770528915414, 'support': 12134.0} | {'precision': 0.877131141644486, 'recall': 0.9351177237518215, 'f1-score': 0.9051967334818115, 'support': 13039.0} | 0.9011 | {'precision': 0.8716887454581927, 'recall': 0.8206855983080162, 'f1-score': 0.8426957732412328, 'support': 31600.0} | {'precision': 0.8989615180207339, 'recall': 0.9010759493670886, 'f1-score': 0.8986163457707048, 'support': 31600.0} |
78
- | No log | 6.0 | 246 | 0.3000 | {'precision': 0.6428049671292915, 'recall': 0.6194274988268419, 'f1-score': 0.6308997490739635, 'support': 4262.0} | {'precision': 0.9450247388675096, 'recall': 0.7939953810623557, 'f1-score': 0.8629518072289157, 'support': 2165.0} | {'precision': 0.996794608366894, 'recall': 0.9995055216746332, 'f1-score': 0.9981482243529073, 'support': 12134.0} | {'precision': 0.8865773302731916, 'recall': 0.9183986502032364, 'f1-score': 0.9022074888872147, 'support': 13039.0} | 0.9007 | {'precision': 0.8678004111592217, 'recall': 0.8328317629417669, 'f1-score': 0.8485518173857503, 'support': 31600.0} | {'precision': 0.9000253454718111, 'recall': 0.9006962025316456, 'f1-score': 0.8997658036424813, 'support': 31600.0} |
79
- | No log | 7.0 | 287 | 0.3219 | {'precision': 0.5949652067130577, 'recall': 0.6820741435945565, 'f1-score': 0.6355487538259729, 'support': 4262.0} | {'precision': 0.9873817034700315, 'recall': 0.7228637413394919, 'f1-score': 0.8346666666666667, 'support': 2165.0} | {'precision': 0.9974464579901153, 'recall': 0.9979396736443052, 'f1-score': 0.9976930048611683, 'support': 12134.0} | {'precision': 0.8966048194626223, 'recall': 0.8931666538845003, 'f1-score': 0.8948824343015215, 'support': 13039.0} | 0.8933 | {'precision': 0.8690995469089566, 'recall': 0.8240110531157135, 'f1-score': 0.8406977149138323, 'support': 31600.0} | {'precision': 0.900862932318002, 'recall': 0.8932594936708861, 'f1-score': 0.8952576298728666, 'support': 31600.0} |
80
- | No log | 8.0 | 328 | 0.3370 | {'precision': 0.6643180674383493, 'recall': 0.6194274988268419, 'f1-score': 0.6410879067508499, 'support': 4262.0} | {'precision': 0.9216867469879518, 'recall': 0.8480369515011548, 'f1-score': 0.8833293240317538, 'support': 2165.0} | {'precision': 0.9963854431939538, 'recall': 0.9995879347288611, 'f1-score': 0.9979841198008804, 'support': 12134.0} | {'precision': 0.889755590223609, 'recall': 0.918552036199095, 'f1-score': 0.9039245283018866, 'support': 13039.0} | 0.9045 | {'precision': 0.868036461960966, 'recall': 0.8464011053139882, 'f1-score': 0.8565814697213427, 'support': 31600.0} | {'precision': 0.9024822632687415, 'recall': 0.9044936708860759, 'f1-score': 0.9031815151675017, 'support': 31600.0} |
81
- | No log | 9.0 | 369 | 0.3583 | {'precision': 0.6696588868940754, 'recall': 0.6126231816048804, 'f1-score': 0.6398725646366867, 'support': 4262.0} | {'precision': 0.923741738688358, 'recall': 0.8392609699769054, 'f1-score': 0.8794772507260408, 'support': 2165.0} | {'precision': 0.9980212713331684, 'recall': 0.9976100214273941, 'f1-score': 0.9978156040060999, 'support': 12134.0} | {'precision': 0.8865123116501287, 'recall': 0.9249942480251553, 'f1-score': 0.9053445428614323, 'support': 13039.0} | 0.9049 | {'precision': 0.8694835521414326, 'recall': 0.8436221052585838, 'f1-score': 0.8556274905575649, 'support': 31600.0} | {'precision': 0.9026332651318208, 'recall': 0.904873417721519, 'f1-score': 0.9032749098634072, 'support': 31600.0} |
82
- | No log | 10.0 | 410 | 0.4008 | {'precision': 0.6432761242667825, 'recall': 0.6947442515251055, 'f1-score': 0.6680203045685279, 'support': 4262.0} | {'precision': 0.8905405405405405, 'recall': 0.9131639722863741, 'f1-score': 0.9017103762827823, 'support': 2165.0} | {'precision': 0.9983533673637411, 'recall': 0.9993406955661777, 'f1-score': 0.9988467874794069, 'support': 12134.0} | {'precision': 0.9138627187079408, 'recall': 0.8852672750977836, 'f1-score': 0.8993377483443709, 'support': 13039.0} | 0.9053 | {'precision': 0.8615081877197512, 'recall': 0.8731290486188602, 'f1-score': 0.866978804168772, 'support': 31600.0} | {'precision': 0.9082132550860689, 'recall': 0.9052848101265822, 'f1-score': 0.9065119405905274, 'support': 31600.0} |
83
- | No log | 11.0 | 451 | 0.4241 | {'precision': 0.6283344541582605, 'recall': 0.6576724542468325, 'f1-score': 0.6426688066032329, 'support': 4262.0} | {'precision': 0.9258312020460358, 'recall': 0.836027713625866, 'f1-score': 0.8786407766990292, 'support': 2165.0} | {'precision': 0.9981066842278564, 'recall': 0.9992582825119499, 'f1-score': 0.9986821513878593, 'support': 12134.0} | {'precision': 0.8980515495550783, 'recall': 0.897844926758187, 'f1-score': 0.897948226270374, 'support': 13039.0} | 0.9002 | {'precision': 0.8625809724968077, 'recall': 0.8477008442857088, 'f1-score': 0.8544849902401238, 'support': 31600.0} | {'precision': 0.9019970461114446, 'recall': 0.9001582278481013, 'f1-score': 0.900875565904306, 'support': 31600.0} |
84
- | No log | 12.0 | 492 | 0.4143 | {'precision': 0.6269706007669366, 'recall': 0.6905208822149226, 'f1-score': 0.6572130415364003, 'support': 4262.0} | {'precision': 0.9402756508422665, 'recall': 0.8508083140877598, 'f1-score': 0.8933074684772067, 'support': 2165.0} | {'precision': 0.998682042833608, 'recall': 0.9991758694577221, 'f1-score': 0.9989288951141139, 'support': 12134.0} | {'precision': 0.9047395955336925, 'recall': 0.8886417670066723, 'f1-score': 0.8966184322525729, 'support': 13039.0} | 0.9018 | {'precision': 0.8676669724941259, 'recall': 0.8572867081917691, 'f1-score': 0.8615169593450734, 'support': 31600.0} | {'precision': 0.9057833221028166, 'recall': 0.9017721518987342, 'f1-score': 0.9033880887258622, 'support': 31600.0} |
85
- | 0.1652 | 13.0 | 533 | 0.4456 | {'precision': 0.6519438693351737, 'recall': 0.664946034725481, 'f1-score': 0.6583807643164131, 'support': 4262.0} | {'precision': 0.9468421052631579, 'recall': 0.8309468822170901, 'f1-score': 0.8851168511685117, 'support': 2165.0} | {'precision': 0.9975333004440059, 'recall': 0.9998351738915444, 'f1-score': 0.9986829107672046, 'support': 12134.0} | {'precision': 0.8987946327041164, 'recall': 0.909272183449651, 'f1-score': 0.9040030499428136, 'support': 13039.0} | 0.9057 | {'precision': 0.8737784769366135, 'recall': 0.8512500685709417, 'f1-score': 0.8615458940487357, 'support': 31600.0} | {'precision': 0.9067072852030946, 'recall': 0.9057278481012658, 'f1-score': 0.905937057207278, 'support': 31600.0} |
86
- | 0.1652 | 14.0 | 574 | 0.4601 | {'precision': 0.6456975960458324, 'recall': 0.674331299859221, 'f1-score': 0.6597038907379777, 'support': 4262.0} | {'precision': 0.9215880893300248, 'recall': 0.8577367205542725, 'f1-score': 0.8885167464114833, 'support': 2165.0} | {'precision': 0.9972053263192504, 'recall': 0.9998351738915444, 'f1-score': 0.9985185185185186, 'support': 12134.0} | {'precision': 0.9043028994447871, 'recall': 0.8993787867167727, 'f1-score': 0.9018341215826509, 'support': 13039.0} | 0.9047 | {'precision': 0.8671984777849736, 'recall': 0.8578204952554527, 'f1-score': 0.8621433193126575, 'support': 31600.0} | {'precision': 0.9062815285811774, 'recall': 0.904746835443038, 'f1-score': 0.9053903656115826, 'support': 31600.0} |
87
- | 0.1652 | 15.0 | 615 | 0.4733 | {'precision': 0.6527744124061061, 'recall': 0.6320976067573909, 'f1-score': 0.6422696388127309, 'support': 4262.0} | {'precision': 0.9225352112676056, 'recall': 0.8471131639722864, 'f1-score': 0.883216951601252, 'support': 2165.0} | {'precision': 0.9979432332373509, 'recall': 0.9996703477830888, 'f1-score': 0.9988060438881798, 'support': 12134.0} | {'precision': 0.8922730682670668, 'recall': 0.912186517370964, 'f1-score': 0.9021199135348325, 'support': 13039.0} | 0.9035 | {'precision': 0.8663814812945323, 'recall': 0.8477669089709325, 'f1-score': 0.8566031369592488, 'support': 31600.0} | {'precision': 0.9026204116235915, 'recall': 0.9035443037974683, 'f1-score': 0.9029041768973552, 'support': 31600.0} |
88
- | 0.1652 | 16.0 | 656 | 0.4725 | {'precision': 0.6448837209302326, 'recall': 0.6506335053965274, 'f1-score': 0.6477458537724831, 'support': 4262.0} | {'precision': 0.9238191975622143, 'recall': 0.8401847575057737, 'f1-score': 0.8800193517174649, 'support': 2165.0} | {'precision': 0.9978613144690301, 'recall': 0.9997527608373167, 'f1-score': 0.9988061421925816, 'support': 12134.0} | {'precision': 0.8974495217853348, 'recall': 0.9067413145179845, 'f1-score': 0.9020714912448022, 'support': 13039.0} | 0.9034 | {'precision': 0.866003438686703, 'recall': 0.8493280845644006, 'f1-score': 0.857160709731833, 'support': 31600.0} | {'precision': 0.9037486229637037, 'recall': 0.9033544303797468, 'f1-score': 0.9034037540807719, 'support': 31600.0} |
89
 
90
 
91
  ### Framework versions
 
17
  name: essays_su_g
18
  type: essays_su_g
19
  config: sep_tok
20
+ split: train[20%:40%]
21
  args: sep_tok
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
+ value: 0.9141753288565909
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
+ - Loss: 0.4313
36
+ - Claim: {'precision': 0.6927914852443154, 'recall': 0.6577859439595773, 'f1-score': 0.6748350612629594, 'support': 4354.0}
37
+ - Majorclaim: {'precision': 0.9005037783375315, 'recall': 0.913154533844189, 'f1-score': 0.9067850348763474, 'support': 2349.0}
38
+ - O: {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0}
39
+ - Premise: {'precision': 0.9048672566371682, 'recall': 0.9174517720951099, 'f1-score': 0.9111160614836266, 'support': 13374.0}
40
+ - Accuracy: 0.9142
41
+ - Macro avg: {'precision': 0.8745007481053241, 'recall': 0.8720781199093074, 'f1-score': 0.873154126750686, 'support': 32613.0}
42
+ - Weighted avg: {'precision': 0.9127462162898813, 'recall': 0.9141753288565909, 'f1-score': 0.9133792098172753, 'support': 32613.0}
43
 
44
  ## Model description
45
 
 
68
 
69
  ### Training results
70
 
71
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
+ |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
+ | No log | 1.0 | 41 | 0.3274 | {'precision': 0.5337112171837709, 'recall': 0.4108865411116215, 'f1-score': 0.4643135219309629, 'support': 4354.0} | {'precision': 0.6779310344827586, 'recall': 0.8369518944231588, 'f1-score': 0.7490950657268052, 'support': 2349.0} | {'precision': 0.9961064759634486, 'recall': 1.0, 'f1-score': 0.9980494407069782, 'support': 12536.0} | {'precision': 0.868321718931475, 'recall': 0.8944220128607746, 'f1-score': 0.8811786372007365, 'support': 13374.0} | 0.8663 | {'precision': 0.7690176116403633, 'recall': 0.7855651120988888, 'f1-score': 0.7731591663913707, 'support': 32613.0} | {'precision': 0.8590551035257559, 'recall': 0.8663109802839358, 'f1-score': 0.8609350954068932, 'support': 32613.0} |
74
+ | No log | 2.0 | 82 | 0.2487 | {'precision': 0.6105208603265094, 'recall': 0.5411116214974736, 'f1-score': 0.5737245829782053, 'support': 4354.0} | {'precision': 0.8533737024221453, 'recall': 0.8399318859088974, 'f1-score': 0.8465994421797898, 'support': 2349.0} | {'precision': 0.9999201915403033, 'recall': 0.9994416081684748, 'f1-score': 0.9996808425756004, 'support': 12536.0} | {'precision': 0.8785221391604371, 'recall': 0.9138627187079408, 'f1-score': 0.8958440225756799, 'support': 13374.0} | 0.8917 | {'precision': 0.8355842233623487, 'recall': 0.8235869585706966, 'f1-score': 0.8289622225773189, 'support': 32613.0} | {'precision': 0.8875950468565349, 'recall': 0.8916689663631068, 'f1-score': 0.8892060198210008, 'support': 32613.0} |
75
+ | No log | 3.0 | 123 | 0.2409 | {'precision': 0.648193359375, 'recall': 0.6097841065686724, 'f1-score': 0.6284023668639053, 'support': 4354.0} | {'precision': 0.8872870249017037, 'recall': 0.8646232439335888, 'f1-score': 0.8758085381630012, 'support': 2349.0} | {'precision': 0.9997607464710104, 'recall': 1.0, 'f1-score': 0.9998803589232302, 'support': 12536.0} | {'precision': 0.8920300971583023, 'recall': 0.9130402273067145, 'f1-score': 0.9024128884454791, 'support': 13374.0} | 0.9025 | {'precision': 0.8568178069765041, 'recall': 0.8468618944522439, 'f1-score': 0.8516260380989039, 'support': 32613.0} | {'precision': 0.900545253284536, 'recall': 0.9024928709410358, 'f1-score': 0.9013800727011249, 'support': 32613.0} |
76
+ | No log | 4.0 | 164 | 0.2301 | {'precision': 0.6427889207258835, 'recall': 0.6182820395039045, 'f1-score': 0.6302973542495903, 'support': 4354.0} | {'precision': 0.8526522593320236, 'recall': 0.9237973605789698, 'f1-score': 0.8868001634654679, 'support': 2349.0} | {'precision': 0.9997606510292005, 'recall': 0.9996011486917677, 'f1-score': 0.999680893498205, 'support': 12536.0} | {'precision': 0.9004945301963135, 'recall': 0.8986092418124719, 'f1-score': 0.8995508982035928, 'support': 13374.0} | 0.9018 | {'precision': 0.8489240903208553, 'recall': 0.8600724476467785, 'f1-score': 0.854082327354214, 'support': 32613.0} | {'precision': 0.9008001866175751, 'recall': 0.9018182933186153, 'f1-score': 0.9011744291494633, 'support': 32613.0} |
77
+ | No log | 5.0 | 205 | 0.2631 | {'precision': 0.6787741203178207, 'recall': 0.5493798805695912, 'f1-score': 0.6072607260726073, 'support': 4354.0} | {'precision': 0.9056947608200455, 'recall': 0.8463175819497658, 'f1-score': 0.875, 'support': 2349.0} | {'precision': 0.9997607273887382, 'recall': 0.9999202297383536, 'f1-score': 0.9998404722022812, 'support': 12536.0} | {'precision': 0.8748955140707718, 'recall': 0.9391356363092568, 'f1-score': 0.9058781103498017, 'support': 13374.0} | 0.9038 | {'precision': 0.864781280649344, 'recall': 0.8336883321417418, 'f1-score': 0.8469948271561726, 'support': 32613.0} | {'precision': 0.8989271945775551, 'recall': 0.903780700947475, 'f1-score': 0.899905013603967, 'support': 32613.0} |
78
+ | No log | 6.0 | 246 | 0.2843 | {'precision': 0.7390988372093024, 'recall': 0.467156637574644, 'f1-score': 0.57247396566282, 'support': 4354.0} | {'precision': 0.886744966442953, 'recall': 0.8999574286930608, 'f1-score': 0.8933023452355799, 'support': 2349.0} | {'precision': 0.9996809952946806, 'recall': 0.9999202297383536, 'f1-score': 0.9998005982053838, 'support': 12536.0} | {'precision': 0.8590842147543178, 'recall': 0.9595483774487812, 'f1-score': 0.9065413958745407, 'support': 13374.0} | 0.9050 | {'precision': 0.8711522534253133, 'recall': 0.8316456683637099, 'f1-score': 0.8430295762445812, 'support': 32613.0} | {'precision': 0.8991013862116997, 'recall': 0.9050378683347131, 'f1-score': 0.896835733694634, 'support': 32613.0} |
79
+ | No log | 7.0 | 287 | 0.2983 | {'precision': 0.6895514223194749, 'recall': 0.5790078089113458, 'f1-score': 0.6294631710362047, 'support': 4354.0} | {'precision': 0.8248984115256742, 'recall': 0.9506172839506173, 'f1-score': 0.8833069620253163, 'support': 2349.0} | {'precision': 0.9992028061224489, 'recall': 0.9998404594767071, 'f1-score': 0.9995215311004785, 'support': 12536.0} | {'precision': 0.8964686998394864, 'recall': 0.9187228951697323, 'f1-score': 0.9074593796159526, 'support': 13374.0} | 0.9068 | {'precision': 0.8525303349517711, 'recall': 0.8620471118771007, 'f1-score': 0.854937760944488, 'support': 32613.0} | {'precision': 0.9031788559978264, 'recall': 0.9068469628675682, 'f1-score': 0.9039933265062536, 'support': 32613.0} |
80
+ | No log | 8.0 | 328 | 0.3043 | {'precision': 0.6570592208961945, 'recall': 0.6701883325677538, 'f1-score': 0.6635588402501421, 'support': 4354.0} | {'precision': 0.8852592895059208, 'recall': 0.9229459344401874, 'f1-score': 0.9037098791162985, 'support': 2349.0} | {'precision': 0.9996012441183507, 'recall': 0.9998404594767071, 'f1-score': 0.9997208374875374, 'support': 12536.0} | {'precision': 0.9098907766990292, 'recall': 0.8969642590100194, 'f1-score': 0.9033812787107464, 'support': 13374.0} | 0.9081 | {'precision': 0.8629526328048738, 'recall': 0.8724847463736669, 'f1-score': 0.8675927088911811, 'support': 32613.0} | {'precision': 0.9088458701337473, 'recall': 0.9081041302548064, 'f1-score': 0.9084190763411705, 'support': 32613.0} |
81
+ | No log | 9.0 | 369 | 0.3104 | {'precision': 0.6748837209302325, 'recall': 0.6665135507579237, 'f1-score': 0.6706725213773977, 'support': 4354.0} | {'precision': 0.9106310885218127, 'recall': 0.9152830991911451, 'f1-score': 0.9129511677282377, 'support': 2349.0} | {'precision': 0.9996012123145638, 'recall': 0.9997606892150607, 'f1-score': 0.9996809444045626, 'support': 12536.0} | {'precision': 0.9039063664827792, 'recall': 0.9066098399880365, 'f1-score': 0.9052560848140958, 'support': 13374.0} | 0.9110 | {'precision': 0.8722555970623471, 'recall': 0.8720417947880416, 'f1-score': 0.8721401795810735, 'support': 32613.0} | {'precision': 0.9105988621342419, 'recall': 0.910986416459694, 'f1-score': 0.9107878958829343, 'support': 32613.0} |
82
+ | No log | 10.0 | 410 | 0.3317 | {'precision': 0.657001414427157, 'recall': 0.6401010564997703, 'f1-score': 0.6484411354118194, 'support': 4354.0} | {'precision': 0.8625536899648575, 'recall': 0.9404001702852277, 'f1-score': 0.8997963340122199, 'support': 2349.0} | {'precision': 0.999680969851651, 'recall': 0.9998404594767071, 'f1-score': 0.9997607083034219, 'support': 12536.0} | {'precision': 0.905666063893912, 'recall': 0.898758785703604, 'f1-score': 0.9021992043833972, 'support': 13374.0} | 0.9061 | {'precision': 0.8562255345343944, 'recall': 0.8697751179913272, 'f1-score': 0.8625493455277146, 'support': 32613.0} | {'precision': 0.905500915362609, 'recall': 0.9060803973875449, 'f1-score': 0.9056494861218845, 'support': 32613.0} |
83
+ | No log | 11.0 | 451 | 0.3783 | {'precision': 0.6712632356562424, 'recall': 0.6260909508497933, 'f1-score': 0.6478906714200832, 'support': 4354.0} | {'precision': 0.8602825745682888, 'recall': 0.9331630481055768, 'f1-score': 0.8952419848887073, 'support': 2349.0} | {'precision': 0.999680969851651, 'recall': 0.9998404594767071, 'f1-score': 0.9997607083034219, 'support': 12536.0} | {'precision': 0.9027922174365067, 'recall': 0.9090025422461493, 'f1-score': 0.9058867362146051, 'support': 13374.0} | 0.9079 | {'precision': 0.8585047493781722, 'recall': 0.8670242501695566, 'f1-score': 0.8621950252067043, 'support': 32613.0} | {'precision': 0.9060628476302188, 'recall': 0.9078894919203998, 'f1-score': 0.9067601525554976, 'support': 32613.0} |
84
+ | No log | 12.0 | 492 | 0.4069 | {'precision': 0.6924342105263158, 'recall': 0.5801561782269178, 'f1-score': 0.6313421644588852, 'support': 4354.0} | {'precision': 0.8779304769603881, 'recall': 0.9246487867177522, 'f1-score': 0.9006842214389384, 'support': 2349.0} | {'precision': 0.9996809952946806, 'recall': 0.9999202297383536, 'f1-score': 0.9998005982053838, 'support': 12536.0} | {'precision': 0.8889048165137615, 'recall': 0.927321668909825, 'f1-score': 0.9077069457659372, 'support': 13374.0} | 0.9087 | {'precision': 0.8647376248237866, 'recall': 0.8580117158982121, 'f1-score': 0.8598834824672862, 'support': 32613.0} | {'precision': 0.9044654345224509, 'recall': 0.908686720019624, 'f1-score': 0.905704596694275, 'support': 32613.0} |
85
+ | 0.1707 | 13.0 | 533 | 0.3896 | {'precision': 0.6659044134461468, 'recall': 0.6688102893890675, 'f1-score': 0.667354188151713, 'support': 4354.0} | {'precision': 0.8783068783068783, 'recall': 0.918688803746275, 'f1-score': 0.8980441115272575, 'support': 2349.0} | {'precision': 0.9998404849258254, 'recall': 1.0, 'f1-score': 0.9999202361011406, 'support': 12536.0} | {'precision': 0.9120422801057002, 'recall': 0.9032451024375654, 'f1-score': 0.9076223749953041, 'support': 13374.0} | 0.9103 | {'precision': 0.8640235141961377, 'recall': 0.8726860488932269, 'f1-score': 0.8682352276938539, 'support': 32613.0} | {'precision': 0.9105002436590061, 'recall': 0.9102505135988717, 'f1-score': 0.9103335319087843, 'support': 32613.0} |
86
+ | 0.1707 | 14.0 | 574 | 0.4518 | {'precision': 0.6987791932059448, 'recall': 0.6047312815801562, 'f1-score': 0.6483624722974637, 'support': 4354.0} | {'precision': 0.8978132884777124, 'recall': 0.9088974031502767, 'f1-score': 0.9033213454622383, 'support': 2349.0} | {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0} | {'precision': 0.8916008614501076, 'recall': 0.9286675639300135, 'f1-score': 0.9097568121886903, 'support': 13374.0} | 0.9114 | {'precision': 0.8720084538340115, 'recall': 0.8605541195997, 'f1-score': 0.8653302448320508, 'support': 32613.0} | {'precision': 0.9079115108212787, 'recall': 0.9113850305093061, 'f1-score': 0.9090381047714349, 'support': 32613.0} |
87
+ | 0.1707 | 15.0 | 615 | 0.4267 | {'precision': 0.6925925925925925, 'recall': 0.6442351860358291, 'f1-score': 0.6675392670157068, 'support': 4354.0} | {'precision': 0.904277848369335, 'recall': 0.9088974031502767, 'f1-score': 0.9065817409766455, 'support': 2349.0} | {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0} | {'precision': 0.9008415660446396, 'recall': 0.9204426499177508, 'f1-score': 0.9105366322719035, 'support': 13374.0} | 0.9133 | {'precision': 0.8743881198022121, 'recall': 0.8683738672105525, 'f1-score': 0.8711344974110167, 'support': 32613.0} | {'precision': 0.911340633421535, 'recall': 0.9132861128997639, 'f1-score': 0.9121529285245232, 'support': 32613.0} |
88
+ | 0.1707 | 16.0 | 656 | 0.4313 | {'precision': 0.6927914852443154, 'recall': 0.6577859439595773, 'f1-score': 0.6748350612629594, 'support': 4354.0} | {'precision': 0.9005037783375315, 'recall': 0.913154533844189, 'f1-score': 0.9067850348763474, 'support': 2349.0} | {'precision': 0.9998404722022812, 'recall': 0.9999202297383536, 'f1-score': 0.999880349379811, 'support': 12536.0} | {'precision': 0.9048672566371682, 'recall': 0.9174517720951099, 'f1-score': 0.9111160614836266, 'support': 13374.0} | 0.9142 | {'precision': 0.8745007481053241, 'recall': 0.8720781199093074, 'f1-score': 0.873154126750686, 'support': 32613.0} | {'precision': 0.9127462162898813, 'recall': 0.9141753288565909, 'f1-score': 0.9133792098172753, 'support': 32613.0} |
89
 
90
 
91
  ### Framework versions
meta_data/meta_s42_e16_cvi1.json CHANGED
@@ -1 +1 @@
1
- {"Claim": {"precision": 0.6925925925925925, "recall": 0.6442351860358291, "f1-score": 0.6675392670157068, "support": 4354.0}, "MajorClaim": {"precision": 0.904277848369335, "recall": 0.9088974031502767, "f1-score": 0.9065817409766455, "support": 2349.0}, "O": {"precision": 0.9998404722022812, "recall": 0.9999202297383536, "f1-score": 0.999880349379811, "support": 12536.0}, "Premise": {"precision": 0.9008415660446396, "recall": 0.9204426499177508, "f1-score": 0.9105366322719035, "support": 13374.0}, "accuracy": 0.9132861128997639, "macro avg": {"precision": 0.8743881198022121, "recall": 0.8683738672105525, "f1-score": 0.8711344974110167, "support": 32613.0}, "weighted avg": {"precision": 0.911340633421535, "recall": 0.9132861128997639, "f1-score": 0.9121529285245232, "support": 32613.0}}
 
1
+ {"Claim": {"precision": 0.6927914852443154, "recall": 0.6577859439595773, "f1-score": 0.6748350612629594, "support": 4354.0}, "MajorClaim": {"precision": 0.9005037783375315, "recall": 0.913154533844189, "f1-score": 0.9067850348763474, "support": 2349.0}, "O": {"precision": 0.9998404722022812, "recall": 0.9999202297383536, "f1-score": 0.999880349379811, "support": 12536.0}, "Premise": {"precision": 0.9048672566371682, "recall": 0.9174517720951099, "f1-score": 0.9111160614836266, "support": 13374.0}, "accuracy": 0.9141753288565909, "macro avg": {"precision": 0.8745007481053241, "recall": 0.8720781199093074, "f1-score": 0.873154126750686, "support": 32613.0}, "weighted avg": {"precision": 0.9127462162898813, "recall": 0.9141753288565909, "f1-score": 0.9133792098172753, "support": 32613.0}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6f68f549d3574ddc833a1bf0e5882e856c15052aa9fd527f955a2646cb3225ed
3
  size 592324828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed403992d376b5299d329739b05850096c2f8dcb6150aa09d40649eb668998a7
3
  size 592324828