--- base_model: allenai/longformer-base-4096 tags: - generated_from_trainer datasets: - essays_su_g metrics: - accuracy model-index: - name: longformer-sep_tok results: - task: name: Token Classification type: token-classification dataset: name: essays_su_g type: essays_su_g config: sep_tok split: train[80%:100%] args: sep_tok metrics: - name: Accuracy type: accuracy value: 0.8985356000673287 --- # longformer-sep_tok This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset. It achieves the following results on the evaluation set: - Loss: 0.2583 - Claim: {'precision': 0.6372016071850626, 'recall': 0.6468330134357005, 'f1-score': 0.6419811882366948, 'support': 4168.0} - Majorclaim: {'precision': 0.892292490118577, 'recall': 0.8392193308550185, 'f1-score': 0.8649425287356322, 'support': 2152.0} - O: {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0} - Premise: {'precision': 0.8961370562556626, 'recall': 0.9011844611944008, 'f1-score': 0.8986536714297514, 'support': 12073.0} - Accuracy: 0.8985 - Macro avg: {'precision': 0.8564077883898256, 'recall': 0.8467429000982954, 'f1-score': 0.8513611920675855, 'support': 29705.0} - Weighted avg: {'precision': 0.8990786876841317, 'recall': 0.8985356000673287, 'f1-score': 0.8987402622673225, 'support': 29705.0} ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 6 ### Training results | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg | |:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:| | No log | 1.0 | 41 | 0.3743 | {'precision': 0.43195876288659796, 'recall': 0.30158349328214973, 'f1-score': 0.35518508053122355, 'support': 4168.0} | {'precision': 0.6687344913151365, 'recall': 0.5009293680297398, 'f1-score': 0.5727948990435707, 'support': 2152.0} | {'precision': 0.9990991802540312, 'recall': 0.980463224893918, 'f1-score': 0.9896934814616517, 'support': 11312.0} | {'precision': 0.8109643516545946, 'recall': 0.9459123664375052, 'f1-score': 0.8732555916650737, 'support': 12073.0} | 0.8364 | {'precision': 0.72768919652759, 'recall': 0.6822221131608281, 'f1-score': 0.69773226317538, 'support': 29705.0} | {'precision': 0.8191248373533424, 'recall': 0.836424844302306, 'f1-score': 0.8231372987329588, 'support': 29705.0} | | No log | 2.0 | 82 | 0.2730 | {'precision': 0.5999298737727911, 'recall': 0.41050863723608444, 'f1-score': 0.4874643874643875, 'support': 4168.0} | {'precision': 0.749693752552062, 'recall': 0.8531598513011153, 'f1-score': 0.7980873723103673, 'support': 2152.0} | {'precision': 0.9993805309734514, 'recall': 0.9983203677510608, 'f1-score': 0.9988501680523616, 'support': 11312.0} | {'precision': 0.8544719169719169, 'recall': 0.9274413981611861, 'f1-score': 0.8894626047583112, 'support': 12073.0} | 0.8765 | {'precision': 0.8008690185675553, 'recall': 0.7973575636123617, 'f1-score': 0.7934661331463568, 'support': 29705.0} | {'precision': 0.8663484493974304, 'recall': 0.8765191045278573, 'f1-score': 0.8680932745470084, 'support': 29705.0} | | No log | 3.0 | 123 | 0.2507 | {'precision': 0.6004319654427646, 'recall': 0.6002879078694817, 'f1-score': 0.6003599280143971, 'support': 4168.0} | {'precision': 0.7873417721518987, 'recall': 0.8671003717472119, 'f1-score': 0.825298540468819, 'support': 2152.0} | {'precision': 0.9997347480106101, 'recall': 0.999557991513437, 'f1-score': 0.9996463619485456, 'support': 11312.0} | {'precision': 0.8981278461797942, 'recall': 0.8821336867390044, 'f1-score': 0.8900589193932557, 'support': 12073.0} | 0.8862 | {'precision': 0.8214090829462669, 'recall': 0.8372699894672837, 'f1-score': 0.8288409374562543, 'support': 29705.0} | {'precision': 0.8870243017021042, 'recall': 0.8862144420131292, 'f1-score': 0.8864508877040778, 'support': 29705.0} | | No log | 4.0 | 164 | 0.2533 | {'precision': 0.6344515441959532, 'recall': 0.5717370441458733, 'f1-score': 0.6014639071176174, 'support': 4168.0} | {'precision': 0.9013713080168776, 'recall': 0.7941449814126395, 'f1-score': 0.8443675889328064, 'support': 2152.0} | {'precision': 0.9999115904871364, 'recall': 0.9998231966053748, 'f1-score': 0.9998673915926268, 'support': 11312.0} | {'precision': 0.8741170930780098, 'recall': 0.9225544603661062, 'f1-score': 0.8976828531130365, 'support': 12073.0} | 0.8935 | {'precision': 0.8524628839444942, 'recall': 0.8220649206324984, 'f1-score': 0.8358454351890219, 'support': 29705.0} | {'precision': 0.8903673007029912, 'recall': 0.8934522807608147, 'f1-score': 0.8911700264460231, 'support': 29705.0} | | No log | 5.0 | 205 | 0.2560 | {'precision': 0.6261571326845479, 'recall': 0.6329174664107485, 'f1-score': 0.6295191504593723, 'support': 4168.0} | {'precision': 0.8881709741550696, 'recall': 0.8303903345724907, 'f1-score': 0.8583093179634967, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9990275813295615, 'f1-score': 0.9995135541502675, 'support': 11312.0} | {'precision': 0.8916988258477707, 'recall': 0.8995278721113228, 'f1-score': 0.8955962394854031, 'support': 12073.0} | 0.8950 | {'precision': 0.8515067331718471, 'recall': 0.8404658136060308, 'f1-score': 0.8457345655146349, 'support': 29705.0} | {'precision': 0.8954265877754937, 'recall': 0.8950008416091567, 'f1-score': 0.8951337550993843, 'support': 29705.0} | | No log | 6.0 | 246 | 0.2583 | {'precision': 0.6372016071850626, 'recall': 0.6468330134357005, 'f1-score': 0.6419811882366948, 'support': 4168.0} | {'precision': 0.892292490118577, 'recall': 0.8392193308550185, 'f1-score': 0.8649425287356322, 'support': 2152.0} | {'precision': 1.0, 'recall': 0.9997347949080623, 'f1-score': 0.9998673798682641, 'support': 11312.0} | {'precision': 0.8961370562556626, 'recall': 0.9011844611944008, 'f1-score': 0.8986536714297514, 'support': 12073.0} | 0.8985 | {'precision': 0.8564077883898256, 'recall': 0.8467429000982954, 'f1-score': 0.8513611920675855, 'support': 29705.0} | {'precision': 0.8990786876841317, 'recall': 0.8985356000673287, 'f1-score': 0.8987402622673225, 'support': 29705.0} | ### Framework versions - Transformers 4.37.2 - Pytorch 2.2.0+cu121 - Datasets 2.17.0 - Tokenizers 0.15.2