--- license: apache-2.0 base_model: google-bert/bert-base-cased tags: - generated_from_trainer model-index: - name: bert_baseline_language_task3_fold2 results: [] --- # bert_baseline_language_task3_fold2 This model is a fine-tuned version of [google-bert/bert-base-cased](https://huggingface.co/google-bert/bert-base-cased) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.3909 - Qwk: 0.6688 - Mse: 0.3915 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 5 ### Training results | Training Loss | Epoch | Step | Validation Loss | Qwk | Mse | |:-------------:|:------:|:----:|:---------------:|:-------:|:------:| | No log | 0.0308 | 2 | 0.8122 | 0.1161 | 0.8118 | | No log | 0.0615 | 4 | 0.6510 | 0.0 | 0.6508 | | No log | 0.0923 | 6 | 0.5802 | 0.0 | 0.5802 | | No log | 0.1231 | 8 | 0.5885 | 0.0 | 0.5887 | | No log | 0.1538 | 10 | 0.6144 | 0.0 | 0.6146 | | No log | 0.1846 | 12 | 0.7590 | -0.0224 | 0.7587 | | No log | 0.2154 | 14 | 0.5770 | 0.0 | 0.5772 | | No log | 0.2462 | 16 | 0.5246 | 0.0 | 0.5248 | | No log | 0.2769 | 18 | 0.5178 | 0.0 | 0.5180 | | No log | 0.3077 | 20 | 0.4920 | 0.0 | 0.4924 | | No log | 0.3385 | 22 | 0.4685 | 0.0196 | 0.4688 | | No log | 0.3692 | 24 | 0.4564 | 0.0292 | 0.4568 | | No log | 0.4 | 26 | 0.4491 | 0.0715 | 0.4497 | | No log | 0.4308 | 28 | 0.4274 | 0.1327 | 0.4279 | | No log | 0.4615 | 30 | 0.4072 | 0.2387 | 0.4077 | | No log | 0.4923 | 32 | 0.3896 | 0.3393 | 0.3902 | | No log | 0.5231 | 34 | 0.3837 | 0.4014 | 0.3841 | | No log | 0.5538 | 36 | 0.4012 | 0.3766 | 0.4015 | | No log | 0.5846 | 38 | 0.3830 | 0.4405 | 0.3836 | | No log | 0.6154 | 40 | 0.3917 | 0.5296 | 0.3925 | | No log | 0.6462 | 42 | 0.4168 | 0.5979 | 0.4178 | | No log | 0.6769 | 44 | 0.3754 | 0.5730 | 0.3762 | | No log | 0.7077 | 46 | 0.3560 | 0.5050 | 0.3566 | | No log | 0.7385 | 48 | 0.3524 | 0.5659 | 0.3531 | | No log | 0.7692 | 50 | 0.3680 | 0.6209 | 0.3688 | | No log | 0.8 | 52 | 0.4054 | 0.6277 | 0.4064 | | No log | 0.8308 | 54 | 0.4243 | 0.6363 | 0.4255 | | No log | 0.8615 | 56 | 0.3582 | 0.5820 | 0.3590 | | No log | 0.8923 | 58 | 0.3258 | 0.4882 | 0.3262 | | No log | 0.9231 | 60 | 0.3386 | 0.5391 | 0.3393 | | No log | 0.9538 | 62 | 0.4628 | 0.6342 | 0.4642 | | No log | 0.9846 | 64 | 0.4746 | 0.6440 | 0.4760 | | No log | 1.0154 | 66 | 0.3604 | 0.5829 | 0.3612 | | No log | 1.0462 | 68 | 0.3423 | 0.5731 | 0.3429 | | No log | 1.0769 | 70 | 0.3299 | 0.5657 | 0.3304 | | No log | 1.1077 | 72 | 0.3770 | 0.6429 | 0.3779 | | No log | 1.1385 | 74 | 0.4051 | 0.6493 | 0.4062 | | No log | 1.1692 | 76 | 0.3676 | 0.6376 | 0.3685 | | No log | 1.2 | 78 | 0.3559 | 0.6044 | 0.3567 | | No log | 1.2308 | 80 | 0.3379 | 0.5606 | 0.3384 | | No log | 1.2615 | 82 | 0.3378 | 0.6022 | 0.3385 | | No log | 1.2923 | 84 | 0.3708 | 0.6298 | 0.3718 | | No log | 1.3231 | 86 | 0.3647 | 0.6367 | 0.3657 | | No log | 1.3538 | 88 | 0.3485 | 0.6188 | 0.3494 | | No log | 1.3846 | 90 | 0.3369 | 0.6202 | 0.3377 | | No log | 1.4154 | 92 | 0.3350 | 0.6310 | 0.3359 | | No log | 1.4462 | 94 | 0.3204 | 0.6162 | 0.3211 | | No log | 1.4769 | 96 | 0.3351 | 0.6265 | 0.3359 | | No log | 1.5077 | 98 | 0.3462 | 0.6457 | 0.3471 | | No log | 1.5385 | 100 | 0.3783 | 0.6460 | 0.3793 | | No log | 1.5692 | 102 | 0.3413 | 0.6203 | 0.3420 | | No log | 1.6 | 104 | 0.3377 | 0.6269 | 0.3383 | | No log | 1.6308 | 106 | 0.3134 | 0.5908 | 0.3138 | | No log | 1.6615 | 108 | 0.3474 | 0.6274 | 0.3480 | | No log | 1.6923 | 110 | 0.3899 | 0.6568 | 0.3909 | | No log | 1.7231 | 112 | 0.3571 | 0.6419 | 0.3578 | | No log | 1.7538 | 114 | 0.3509 | 0.6403 | 0.3516 | | No log | 1.7846 | 116 | 0.3315 | 0.6109 | 0.3320 | | No log | 1.8154 | 118 | 0.3362 | 0.6151 | 0.3368 | | No log | 1.8462 | 120 | 0.4204 | 0.6775 | 0.4215 | | No log | 1.8769 | 122 | 0.5712 | 0.6576 | 0.5728 | | No log | 1.9077 | 124 | 0.5938 | 0.6452 | 0.5955 | | No log | 1.9385 | 126 | 0.5023 | 0.6637 | 0.5037 | | No log | 1.9692 | 128 | 0.3689 | 0.6703 | 0.3699 | | No log | 2.0 | 130 | 0.3049 | 0.6244 | 0.3056 | | No log | 2.0308 | 132 | 0.2948 | 0.5986 | 0.2953 | | No log | 2.0615 | 134 | 0.3149 | 0.6378 | 0.3156 | | No log | 2.0923 | 136 | 0.3613 | 0.6727 | 0.3624 | | No log | 2.1231 | 138 | 0.4256 | 0.6760 | 0.4269 | | No log | 2.1538 | 140 | 0.4153 | 0.6726 | 0.4165 | | No log | 2.1846 | 142 | 0.3609 | 0.6697 | 0.3619 | | No log | 2.2154 | 144 | 0.3059 | 0.6049 | 0.3064 | | No log | 2.2462 | 146 | 0.2949 | 0.5649 | 0.2951 | | No log | 2.2769 | 148 | 0.3011 | 0.5764 | 0.3015 | | No log | 2.3077 | 150 | 0.3428 | 0.6515 | 0.3436 | | No log | 2.3385 | 152 | 0.3738 | 0.6810 | 0.3748 | | No log | 2.3692 | 154 | 0.3593 | 0.6789 | 0.3602 | | No log | 2.4 | 156 | 0.3142 | 0.6219 | 0.3148 | | No log | 2.4308 | 158 | 0.3166 | 0.6136 | 0.3171 | | No log | 2.4615 | 160 | 0.3548 | 0.6757 | 0.3555 | | No log | 2.4923 | 162 | 0.3589 | 0.6783 | 0.3597 | | No log | 2.5231 | 164 | 0.3387 | 0.6486 | 0.3393 | | No log | 2.5538 | 166 | 0.3463 | 0.6598 | 0.3469 | | No log | 2.5846 | 168 | 0.3630 | 0.6700 | 0.3637 | | No log | 2.6154 | 170 | 0.3479 | 0.6690 | 0.3486 | | No log | 2.6462 | 172 | 0.3218 | 0.6237 | 0.3223 | | No log | 2.6769 | 174 | 0.3360 | 0.6367 | 0.3366 | | No log | 2.7077 | 176 | 0.3513 | 0.6676 | 0.3520 | | No log | 2.7385 | 178 | 0.3358 | 0.6439 | 0.3364 | | No log | 2.7692 | 180 | 0.3264 | 0.6396 | 0.3270 | | No log | 2.8 | 182 | 0.3054 | 0.5999 | 0.3057 | | No log | 2.8308 | 184 | 0.3175 | 0.6373 | 0.3180 | | No log | 2.8615 | 186 | 0.3453 | 0.6678 | 0.3459 | | No log | 2.8923 | 188 | 0.4004 | 0.6836 | 0.4013 | | No log | 2.9231 | 190 | 0.4915 | 0.6815 | 0.4928 | | No log | 2.9538 | 192 | 0.4985 | 0.6860 | 0.4998 | | No log | 2.9846 | 194 | 0.4225 | 0.6811 | 0.4235 | | No log | 3.0154 | 196 | 0.3205 | 0.6513 | 0.3210 | | No log | 3.0462 | 198 | 0.2921 | 0.5891 | 0.2923 | | No log | 3.0769 | 200 | 0.2967 | 0.6134 | 0.2969 | | No log | 3.1077 | 202 | 0.3214 | 0.6572 | 0.3219 | | No log | 3.1385 | 204 | 0.3733 | 0.6819 | 0.3740 | | No log | 3.1692 | 206 | 0.4341 | 0.6954 | 0.4351 | | No log | 3.2 | 208 | 0.4243 | 0.6916 | 0.4252 | | No log | 3.2308 | 210 | 0.4523 | 0.6935 | 0.4534 | | No log | 3.2615 | 212 | 0.4334 | 0.6896 | 0.4343 | | No log | 3.2923 | 214 | 0.3839 | 0.6785 | 0.3846 | | No log | 3.3231 | 216 | 0.3687 | 0.6729 | 0.3693 | | No log | 3.3538 | 218 | 0.3710 | 0.6801 | 0.3717 | | No log | 3.3846 | 220 | 0.3705 | 0.6832 | 0.3712 | | No log | 3.4154 | 222 | 0.4216 | 0.6858 | 0.4225 | | No log | 3.4462 | 224 | 0.4302 | 0.6801 | 0.4312 | | No log | 3.4769 | 226 | 0.3861 | 0.6768 | 0.3870 | | No log | 3.5077 | 228 | 0.3411 | 0.6621 | 0.3417 | | No log | 3.5385 | 230 | 0.3246 | 0.6410 | 0.3251 | | No log | 3.5692 | 232 | 0.3350 | 0.6441 | 0.3355 | | No log | 3.6 | 234 | 0.3310 | 0.6431 | 0.3315 | | No log | 3.6308 | 236 | 0.3315 | 0.6407 | 0.3320 | | No log | 3.6615 | 238 | 0.3251 | 0.6355 | 0.3256 | | No log | 3.6923 | 240 | 0.3472 | 0.6496 | 0.3478 | | No log | 3.7231 | 242 | 0.4075 | 0.6778 | 0.4084 | | No log | 3.7538 | 244 | 0.4415 | 0.6789 | 0.4426 | | No log | 3.7846 | 246 | 0.4342 | 0.6772 | 0.4352 | | No log | 3.8154 | 248 | 0.3928 | 0.6756 | 0.3937 | | No log | 3.8462 | 250 | 0.3569 | 0.6567 | 0.3575 | | No log | 3.8769 | 252 | 0.3368 | 0.6401 | 0.3374 | | No log | 3.9077 | 254 | 0.3360 | 0.6429 | 0.3365 | | No log | 3.9385 | 256 | 0.3304 | 0.6407 | 0.3309 | | No log | 3.9692 | 258 | 0.3356 | 0.6400 | 0.3360 | | No log | 4.0 | 260 | 0.3335 | 0.6400 | 0.3339 | | No log | 4.0308 | 262 | 0.3577 | 0.6524 | 0.3582 | | No log | 4.0615 | 264 | 0.3824 | 0.6662 | 0.3830 | | No log | 4.0923 | 266 | 0.4111 | 0.6804 | 0.4118 | | No log | 4.1231 | 268 | 0.4055 | 0.6870 | 0.4061 | | No log | 4.1538 | 270 | 0.3957 | 0.6760 | 0.3963 | | No log | 4.1846 | 272 | 0.3908 | 0.6743 | 0.3914 | | No log | 4.2154 | 274 | 0.4130 | 0.6824 | 0.4137 | | No log | 4.2462 | 276 | 0.4355 | 0.6956 | 0.4362 | | No log | 4.2769 | 278 | 0.4466 | 0.6970 | 0.4474 | | No log | 4.3077 | 280 | 0.4483 | 0.6997 | 0.4491 | | No log | 4.3385 | 282 | 0.4194 | 0.6886 | 0.4201 | | No log | 4.3692 | 284 | 0.3988 | 0.6748 | 0.3994 | | No log | 4.4 | 286 | 0.3800 | 0.6654 | 0.3804 | | No log | 4.4308 | 288 | 0.3798 | 0.6685 | 0.3803 | | No log | 4.4615 | 290 | 0.3838 | 0.6724 | 0.3844 | | No log | 4.4923 | 292 | 0.3791 | 0.6732 | 0.3796 | | No log | 4.5231 | 294 | 0.3917 | 0.6687 | 0.3923 | | No log | 4.5538 | 296 | 0.4094 | 0.6720 | 0.4101 | | No log | 4.5846 | 298 | 0.4175 | 0.6706 | 0.4181 | | No log | 4.6154 | 300 | 0.4404 | 0.6876 | 0.4412 | | No log | 4.6462 | 302 | 0.4473 | 0.6909 | 0.4481 | | No log | 4.6769 | 304 | 0.4364 | 0.6785 | 0.4372 | | No log | 4.7077 | 306 | 0.4162 | 0.6762 | 0.4169 | | No log | 4.7385 | 308 | 0.4050 | 0.6752 | 0.4056 | | No log | 4.7692 | 310 | 0.3955 | 0.6712 | 0.3961 | | No log | 4.8 | 312 | 0.3946 | 0.6712 | 0.3952 | | No log | 4.8308 | 314 | 0.3947 | 0.6712 | 0.3953 | | No log | 4.8615 | 316 | 0.3947 | 0.6712 | 0.3953 | | No log | 4.8923 | 318 | 0.3938 | 0.6712 | 0.3944 | | No log | 4.9231 | 320 | 0.3910 | 0.6688 | 0.3916 | | No log | 4.9538 | 322 | 0.3909 | 0.6688 | 0.3915 | | No log | 4.9846 | 324 | 0.3909 | 0.6688 | 0.3915 | ### Framework versions - Transformers 4.42.3 - Pytorch 2.1.2 - Datasets 2.20.0 - Tokenizers 0.19.1