Hugging Face's logo

gngpostalsrvc
/
BERiT_2000_custom_architecture_relu_40_epochs

Fill-Mask
Transformers
PyTorch
TensorBoard
roberta
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
1
BERiT_2000_custom_architecture_relu_40_epochs / README.md
gngpostalsrvc's picture
gngpostalsrvc
update model card README.md
fff30fe
preview code
|
raw
history blame contribute delete
12.2 kB
metadata
license: mit
tags:
  - generated_from_trainer
model-index:
  - name: BERiT_2000_custom_architecture_relu_40_epochs
    results: []

BERiT_2000_custom_architecture_relu_40_epochs

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 6.3968

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0005
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 40
  • label_smoothing_factor: 0.2

Training results

Training Loss Epoch Step Validation Loss
14.8756 0.19 500 8.3404
7.8098 0.39 1000 7.3110
7.2696 0.58 1500 7.1646
7.1277 0.77 2000 7.0953
7.0939 0.97 2500 7.0701
7.0621 1.16 3000 7.0003
7.0236 1.36 3500 6.9189
6.9898 1.55 4000 6.8536
6.9625 1.74 4500 6.8450
6.9125 1.94 5000 6.7799
6.9115 2.13 5500 6.8028
6.8954 2.32 6000 6.7288
6.8289 2.52 6500 6.7664
6.855 2.71 7000 6.7064
6.8134 2.9 7500 6.7155
6.7907 3.1 8000 6.7050
6.791 3.29 8500 6.6695
6.7659 3.49 9000 6.6815
6.775 3.68 9500 6.6449
6.7508 3.87 10000 6.6684
6.7627 4.07 10500 6.6397
6.7229 4.26 11000 6.6417
6.7336 4.45 11500 6.6824
6.7138 4.65 12000 6.6252
6.7123 4.84 12500 6.6374
6.703 5.03 13000 6.6400
6.7054 5.23 13500 6.6264
6.6978 5.42 14000 6.6094
6.6944 5.62 14500 6.6627
6.6857 5.81 15000 6.6363
6.694 6.0 15500 6.6026
6.6882 6.2 16000 6.6123
6.6694 6.39 16500 6.5918
6.6781 6.58 17000 6.6201
6.6605 6.78 17500 6.6316
6.6435 6.97 18000 6.5789
6.6658 7.16 18500 6.5999
6.6551 7.36 19000 6.5425
6.6603 7.55 19500 6.5790
6.6589 7.75 20000 6.5767
6.675 7.94 20500 6.6005
6.6362 8.13 21000 6.5962
6.6391 8.33 21500 6.5716
6.6379 8.52 22000 6.5830
6.6164 8.71 22500 6.6137
6.638 8.91 23000 6.5877
6.6255 9.1 23500 6.6197
6.6284 9.3 24000 6.5573
6.6198 9.49 24500 6.5717
6.6025 9.68 25000 6.5627
6.6334 9.88 25500 6.5902
6.6305 10.07 26000 6.5628
6.5797 10.26 26500 6.5625
6.5906 10.46 27000 6.5808
6.5904 10.65 27500 6.5690
6.5935 10.84 28000 6.5845
6.6231 11.04 28500 6.5282
6.5923 11.23 29000 6.6107
6.6136 11.43 29500 6.5475
6.5954 11.62 30000 6.5823
6.5821 11.81 30500 6.5721
6.5993 12.01 31000 6.5492
6.5584 12.2 31500 6.4938
6.5886 12.39 32000 6.6026
6.5625 12.59 32500 6.5902
6.572 12.78 33000 6.5436
6.5807 12.97 33500 6.5588
6.5853 13.17 34000 6.5555
6.5727 13.36 34500 6.5606
6.5456 13.56 35000 6.5386
6.5538 13.75 35500 6.5712
6.5456 13.94 36000 6.5582
6.5734 14.14 36500 6.4951
6.5639 14.33 37000 6.5323
6.5712 14.52 37500 6.5049
6.5739 14.72 38000 6.5523
6.5534 14.91 38500 6.5188
6.5401 15.1 39000 6.5968
6.5456 15.3 39500 6.5413
6.5555 15.49 40000 6.5347
6.538 15.69 40500 6.5180
6.537 15.88 41000 6.5372
6.537 16.07 41500 6.5514
6.5445 16.27 42000 6.5242
6.5285 16.46 42500 6.5071
6.5046 16.65 43000 6.5342
6.5609 16.85 43500 6.5329
6.527 17.04 44000 6.5569
6.5199 17.23 44500 6.5438
6.5328 17.43 45000 6.5380
6.5183 17.62 45500 6.5273
6.5349 17.82 46000 6.5209
6.5283 18.01 46500 6.4884
6.5111 18.2 47000 6.5036
6.4895 18.4 47500 6.5675
6.5308 18.59 48000 6.5378
6.5159 18.78 48500 6.4792
6.4875 18.98 49000 6.4846
6.5076 19.17 49500 6.5203
6.4991 19.36 50000 6.5007
6.5269 19.56 50500 6.4796
6.4887 19.75 51000 6.5197
6.4995 19.95 51500 6.5009
6.4762 20.14 52000 6.5049
6.4872 20.33 52500 6.4880
6.5117 20.53 53000 6.4917
6.5035 20.72 53500 6.4791
6.4784 20.91 54000 6.4771
6.4749 21.11 54500 6.5230
6.4867 21.3 55000 6.4954
6.4921 21.49 55500 6.5079
6.4587 21.69 56000 6.5309
6.4839 21.88 56500 6.4476
6.5011 22.08 57000 6.5025
6.471 22.27 57500 6.5122
6.4689 22.46 58000 6.4689
6.4764 22.66 58500 6.5073
6.4764 22.85 59000 6.4741
6.4751 23.04 59500 6.4978
6.4823 23.24 60000 6.4857
6.4594 23.43 60500 6.4817
6.4795 23.63 61000 6.5292
6.4565 23.82 61500 6.4684
6.4627 24.01 62000 6.4900
6.4542 24.21 62500 6.4373
6.4692 24.4 63000 6.4787
6.4772 24.59 63500 6.4553
6.4613 24.79 64000 6.4695
6.4673 24.98 64500 6.5077
6.466 25.17 65000 6.4919
6.4595 25.37 65500 6.4451
6.444 25.56 66000 6.4750
6.438 25.76 66500 6.4672
6.4499 25.95 67000 6.4358
6.4578 26.14 67500 6.4762
6.4701 26.34 68000 6.4462
6.4296 26.53 68500 6.4879
6.4305 26.72 69000 6.4519
6.443 26.92 69500 6.4530
6.4571 27.11 70000 6.4564
6.4477 27.3 70500 6.4557
6.443 27.5 71000 6.4862
6.4429 27.69 71500 6.4498
6.4374 27.89 72000 6.4225
6.4363 28.08 72500 6.4723
6.4127 28.27 73000 6.4733
6.4116 28.47 73500 6.4499
6.4312 28.66 74000 6.4600
6.4251 28.85 74500 6.4451
6.4318 29.05 75000 6.4337
6.4432 29.24 75500 6.4713
6.4183 29.43 76000 6.4699
6.4109 29.63 76500 6.4591
6.3939 29.82 77000 6.4768
6.4194 30.02 77500 6.4786
6.4262 30.21 78000 6.4407
6.4392 30.4 78500 6.4202
6.4311 30.6 79000 6.4361
6.3963 30.79 79500 6.4346
6.3872 30.98 80000 6.3810
6.4277 31.18 80500 6.4451
6.4112 31.37 81000 6.4243
6.4202 31.56 81500 6.4502
6.444 31.76 82000 6.4572
6.4066 31.95 82500 6.4033
6.4101 32.15 83000 6.4154
6.3985 32.34 83500 6.4377
6.4294 32.53 84000 6.4392
6.397 32.73 84500 6.4387
6.4217 32.92 85000 6.4305
6.4061 33.11 85500 6.4541
6.4014 33.31 86000 6.4173
6.4223 33.5 86500 6.4403
6.3953 33.69 87000 6.4333
6.4135 33.89 87500 6.4183
6.3955 34.08 88000 6.3958
6.4064 34.28 88500 6.3913
6.3997 34.47 89000 6.4330
6.4212 34.66 89500 6.3955
6.3957 34.86 90000 6.4438
6.3936 35.05 90500 6.4382
6.3927 35.24 91000 6.4055
6.3972 35.44 91500 6.4006
6.4137 35.63 92000 6.4245
6.3947 35.82 92500 6.4057
6.3798 36.02 93000 6.4006
6.4011 36.21 93500 6.3943
6.4012 36.41 94000 6.3766
6.3961 36.6 94500 6.4260
6.3819 36.79 95000 6.3801
6.3795 36.99 95500 6.4019
6.3954 37.18 96000 6.4387
6.3874 37.37 96500 6.4477
6.3844 37.57 97000 6.4177
6.3898 37.76 97500 6.4213
6.3855 37.96 98000 6.3838
6.3825 38.15 98500 6.4048
6.3615 38.34 99000 6.4636
6.392 38.54 99500 6.4197
6.3773 38.73 100000 6.4505
6.3834 38.92 100500 6.3889
6.3846 39.12 101000 6.4394
6.376 39.31 101500 6.3923
6.3699 39.5 102000 6.4025
6.3826 39.7 102500 6.3951
6.373 39.89 103000 6.3968

Framework versions

  • Transformers 4.24.0
  • Pytorch 1.12.1+cu113
  • Datasets 2.7.1
  • Tokenizers 0.13.2