Edit model card

pos_final_mono_en

This model is a fine-tuned version of FacebookAI/roberta-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0681
  • Precision: 0.9696
  • Recall: 0.9714
  • F1: 0.9705
  • Accuracy: 0.9796

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 256
  • eval_batch_size: 256
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 1024
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 40.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
No log 0.99 60 2.7933 0.3216 0.0997 0.1522 0.2833
No log 1.99 120 0.3818 0.9075 0.8989 0.9032 0.9224
No log 2.99 180 0.1156 0.9602 0.9607 0.9605 0.9721
No log 3.99 240 0.0911 0.9634 0.9650 0.9642 0.9748
No log 4.99 300 0.0794 0.9664 0.9679 0.9671 0.9772
No log 5.99 360 0.0741 0.9670 0.9697 0.9683 0.9781
No log 6.99 420 0.0695 0.9683 0.9702 0.9693 0.9787
No log 7.99 480 0.0688 0.9686 0.9700 0.9693 0.9789
0.7281 8.99 540 0.0675 0.9688 0.9703 0.9695 0.9789
0.7281 9.99 600 0.0670 0.9687 0.9705 0.9696 0.9791
0.7281 10.99 660 0.0658 0.9696 0.9702 0.9699 0.9792
0.7281 11.99 720 0.0670 0.9684 0.9715 0.9700 0.9793
0.7281 12.99 780 0.0672 0.9689 0.9711 0.9700 0.9792
0.7281 13.99 840 0.0678 0.9698 0.9708 0.9703 0.9796
0.7281 14.99 900 0.0681 0.9696 0.9714 0.9705 0.9796
0.7281 15.99 960 0.0706 0.9696 0.9711 0.9703 0.9795
0.0484 16.99 1020 0.0725 0.9694 0.9705 0.9699 0.9793
0.0484 17.99 1080 0.0735 0.9689 0.9705 0.9697 0.9791
0.0484 18.99 1140 0.0745 0.9690 0.9705 0.9698 0.9792
0.0484 19.99 1200 0.0769 0.9690 0.9706 0.9698 0.9791
0.0484 20.99 1260 0.0797 0.9691 0.9703 0.9697 0.9791
0.0484 21.99 1320 0.0808 0.9689 0.9705 0.9697 0.9791
0.0484 22.99 1380 0.0838 0.9691 0.9702 0.9697 0.9791
0.0484 23.99 1440 0.0861 0.9685 0.9704 0.9695 0.9789
0.0289 24.99 1500 0.0879 0.9684 0.9698 0.9691 0.9787
0.0289 25.99 1560 0.0887 0.9684 0.9703 0.9694 0.9789
0.0289 26.99 1620 0.0910 0.9684 0.9698 0.9691 0.9787
0.0289 27.99 1680 0.0924 0.9684 0.9697 0.9691 0.9787
0.0289 28.99 1740 0.0950 0.9693 0.9692 0.9693 0.9788
0.0289 29.99 1800 0.0962 0.9692 0.9697 0.9694 0.9789
0.0289 30.99 1860 0.0977 0.9687 0.9699 0.9693 0.9787
0.0289 31.99 1920 0.0979 0.9688 0.9699 0.9694 0.9788
0.0289 32.99 1980 0.1000 0.9687 0.9698 0.9692 0.9788
0.018 33.99 2040 0.1021 0.9688 0.9698 0.9693 0.9788
0.018 34.99 2100 0.1037 0.9687 0.9701 0.9694 0.9788
0.018 35.99 2160 0.1035 0.9688 0.9703 0.9696 0.9790
0.018 36.99 2220 0.1042 0.9688 0.9700 0.9694 0.9789
0.018 37.99 2280 0.1053 0.9685 0.9699 0.9692 0.9787
0.018 38.99 2340 0.1052 0.9689 0.9700 0.9695 0.9789
0.018 39.99 2400 0.1054 0.9688 0.9700 0.9694 0.9788

Framework versions

  • Transformers 4.25.1
  • Pytorch 1.12.0
  • Datasets 2.18.0
  • Tokenizers 0.13.2
Downloads last month
4
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Collection including pranaydeeps/lettuce_pos_en_mono