--- language: - he license: apache-2.0 base_model: openai/whisper-tiny tags: - hf-asr-leaderboard - generated_from_trainer metrics: - wer model-index: - name: he-cantillation results: [] --- # he-cantillation This model is a fine-tuned version of [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.2642 - Wer: 16.6210 - Avg Precision Exact: 0.8369 - Avg Recall Exact: 0.8382 - Avg F1 Exact: 0.8371 - Avg Precision Letter Shift: 0.8586 - Avg Recall Letter Shift: 0.8599 - Avg F1 Letter Shift: 0.8588 - Avg Precision Word Level: 0.8633 - Avg Recall Word Level: 0.8652 - Avg F1 Word Level: 0.8637 - Avg Precision Word Shift: 0.9480 - Avg Recall Word Shift: 0.9510 - Avg F1 Word Shift: 0.9488 - Precision Median Exact: 0.9231 - Recall Median Exact: 0.9231 - F1 Median Exact: 0.9286 - Precision Max Exact: 1.0 - Recall Max Exact: 1.0 - F1 Max Exact: 1.0 - Precision Min Exact: 0.0 - Recall Min Exact: 0.0 - F1 Min Exact: 0.0 - Precision Min Letter Shift: 0.0 - Recall Min Letter Shift: 0.0 - F1 Min Letter Shift: 0.0 - Precision Min Word Level: 0.0 - Recall Min Word Level: 0.0 - F1 Min Word Level: 0.0 - Precision Min Word Shift: 0.1429 - Recall Min Word Shift: 0.125 - F1 Min Word Shift: 0.1333 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 8 - eval_batch_size: 32 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 500 - training_steps: 500000 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Wer | Avg Precision Exact | Avg Recall Exact | Avg F1 Exact | Avg Precision Letter Shift | Avg Recall Letter Shift | Avg F1 Letter Shift | Avg Precision Word Level | Avg Recall Word Level | Avg F1 Word Level | Avg Precision Word Shift | Avg Recall Word Shift | Avg F1 Word Shift | Precision Median Exact | Recall Median Exact | F1 Median Exact | Precision Max Exact | Recall Max Exact | F1 Max Exact | Precision Min Exact | Recall Min Exact | F1 Min Exact | Precision Min Letter Shift | Recall Min Letter Shift | F1 Min Letter Shift | Precision Min Word Level | Recall Min Word Level | F1 Min Word Level | Precision Min Word Shift | Recall Min Word Shift | F1 Min Word Shift | |:-------------:|:-------:|:------:|:---------------:|:--------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:| | No log | 0.0001 | 1 | 9.1305 | 104.1002 | 0.0008 | 0.0017 | 0.0007 | 0.0055 | 0.0068 | 0.0048 | 0.0027 | 0.0187 | 0.0046 | 0.0242 | 0.0322 | 0.0233 | 0.0 | 0.0 | 0.0 | 0.1429 | 1.0 | 0.1818 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.1404 | 0.5167 | 10000 | 0.2062 | 31.3792 | 0.7018 | 0.7031 | 0.7017 | 0.7345 | 0.7359 | 0.7344 | 0.7437 | 0.7448 | 0.7434 | 0.8802 | 0.8850 | 0.8815 | 0.8 | 0.8 | 0.8 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0769 | 0.125 | 0.1111 | | 0.0565 | 1.0334 | 20000 | 0.1776 | 25.7434 | 0.7505 | 0.7534 | 0.7513 | 0.7792 | 0.7821 | 0.7799 | 0.7865 | 0.7900 | 0.7875 | 0.9069 | 0.9122 | 0.9086 | 0.8462 | 0.8462 | 0.8485 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0769 | 0.0909 | 0.0833 | | 0.0343 | 1.5501 | 30000 | 0.1768 | 24.2487 | 0.7700 | 0.7691 | 0.7689 | 0.7984 | 0.7974 | 0.7972 | 0.8052 | 0.8043 | 0.8040 | 0.9143 | 0.9158 | 0.9142 | 0.875 | 0.875 | 0.875 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0202 | 2.0668 | 40000 | 0.1824 | 22.5935 | 0.7783 | 0.7853 | 0.7812 | 0.8052 | 0.8126 | 0.8082 | 0.8107 | 0.8185 | 0.8139 | 0.9196 | 0.9301 | 0.9240 | 0.8889 | 0.9 | 0.8889 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.0136 | 2.5834 | 50000 | 0.1848 | 21.7408 | 0.7886 | 0.7912 | 0.7893 | 0.8142 | 0.8169 | 0.8150 | 0.8199 | 0.8229 | 0.8208 | 0.9234 | 0.9281 | 0.9249 | 0.9 | 0.9 | 0.9 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.0338 | 3.1001 | 60000 | 0.1936 | 21.6149 | 0.7881 | 0.7909 | 0.7889 | 0.8137 | 0.8166 | 0.8146 | 0.8188 | 0.8233 | 0.8203 | 0.9222 | 0.9295 | 0.9250 | 0.9 | 0.9 | 0.9 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.0148 | 3.6168 | 70000 | 0.1932 | 20.5198 | 0.8045 | 0.8086 | 0.8059 | 0.8295 | 0.8339 | 0.8311 | 0.8346 | 0.8394 | 0.8364 | 0.9261 | 0.9333 | 0.9289 | 0.9 | 0.9091 | 0.9 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.007 | 4.1335 | 80000 | 0.1995 | 20.3562 | 0.8039 | 0.8048 | 0.8038 | 0.8285 | 0.8296 | 0.8285 | 0.8340 | 0.8356 | 0.8342 | 0.9314 | 0.9354 | 0.9326 | 0.9091 | 0.9091 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0073 | 4.6502 | 90000 | 0.2057 | 20.7275 | 0.8005 | 0.8036 | 0.8015 | 0.8252 | 0.8284 | 0.8262 | 0.8304 | 0.8335 | 0.8314 | 0.9252 | 0.9312 | 0.9274 | 0.9091 | 0.9091 | 0.9 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0033 | 5.1669 | 100000 | 0.2070 | 19.5727 | 0.8112 | 0.8105 | 0.8103 | 0.8356 | 0.8350 | 0.8347 | 0.8410 | 0.8407 | 0.8403 | 0.9364 | 0.9383 | 0.9366 | 0.9091 | 0.9091 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0058 | 5.6836 | 110000 | 0.2166 | 19.9157 | 0.8124 | 0.8128 | 0.8121 | 0.8371 | 0.8377 | 0.8368 | 0.8426 | 0.8434 | 0.8424 | 0.9334 | 0.9368 | 0.9344 | 0.9091 | 0.9091 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.0022 | 6.2003 | 120000 | 0.2149 | 19.3398 | 0.8147 | 0.8156 | 0.8146 | 0.8385 | 0.8395 | 0.8384 | 0.8438 | 0.8450 | 0.8438 | 0.9363 | 0.9395 | 0.9371 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0032 | 6.7170 | 130000 | 0.2221 | 19.3744 | 0.8092 | 0.8134 | 0.8107 | 0.8333 | 0.8378 | 0.8350 | 0.8392 | 0.8439 | 0.8409 | 0.9326 | 0.9391 | 0.9351 | 0.9091 | 0.9167 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0909 | 0.0667 | 0.0769 | | 0.0033 | 7.2336 | 140000 | 0.2266 | 19.7017 | 0.8094 | 0.8137 | 0.8109 | 0.8344 | 0.8390 | 0.8361 | 0.8397 | 0.8447 | 0.8416 | 0.9317 | 0.9391 | 0.9347 | 0.9091 | 0.9167 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0028 | 7.7503 | 150000 | 0.2305 | 19.2454 | 0.8130 | 0.8163 | 0.8141 | 0.8365 | 0.8400 | 0.8377 | 0.8414 | 0.8456 | 0.8430 | 0.9331 | 0.9380 | 0.9348 | 0.9091 | 0.9167 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0018 | 8.2670 | 160000 | 0.2335 | 19.0220 | 0.8196 | 0.8232 | 0.8209 | 0.8429 | 0.8468 | 0.8443 | 0.8480 | 0.8524 | 0.8496 | 0.9366 | 0.9435 | 0.9393 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.005 | 8.7837 | 170000 | 0.2321 | 18.8961 | 0.8197 | 0.8217 | 0.8202 | 0.8432 | 0.8453 | 0.8437 | 0.8482 | 0.8508 | 0.8489 | 0.9372 | 0.9415 | 0.9387 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0015 | 9.3004 | 180000 | 0.2320 | 18.7734 | 0.8198 | 0.8223 | 0.8205 | 0.8429 | 0.8456 | 0.8437 | 0.8477 | 0.8508 | 0.8487 | 0.9381 | 0.9431 | 0.9399 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0032 | 9.8171 | 190000 | 0.2357 | 18.4178 | 0.8207 | 0.8226 | 0.8211 | 0.8440 | 0.8460 | 0.8445 | 0.8485 | 0.8513 | 0.8493 | 0.9390 | 0.9434 | 0.9405 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0025 | 10.3338 | 200000 | 0.2467 | 19.3430 | 0.8115 | 0.8124 | 0.8114 | 0.8346 | 0.8355 | 0.8345 | 0.8395 | 0.8409 | 0.8397 | 0.9325 | 0.9363 | 0.9337 | 0.9091 | 0.9091 | 0.9091 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0007 | 10.8505 | 210000 | 0.2415 | 18.6381 | 0.8175 | 0.8200 | 0.8182 | 0.8411 | 0.8437 | 0.8419 | 0.8455 | 0.8484 | 0.8464 | 0.9381 | 0.9430 | 0.9399 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0009 | 11.3672 | 220000 | 0.2405 | 18.0843 | 0.8247 | 0.8244 | 0.8241 | 0.8475 | 0.8474 | 0.8469 | 0.8522 | 0.8526 | 0.8519 | 0.9410 | 0.9437 | 0.9416 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.0027 | 11.8838 | 230000 | 0.2407 | 18.3517 | 0.8176 | 0.8208 | 0.8187 | 0.8405 | 0.8438 | 0.8416 | 0.8455 | 0.8491 | 0.8468 | 0.9377 | 0.9438 | 0.9401 | 0.9167 | 0.9167 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0004 | 12.4005 | 240000 | 0.2424 | 18.4808 | 0.8130 | 0.8141 | 0.8130 | 0.8361 | 0.8374 | 0.8362 | 0.8412 | 0.8428 | 0.8415 | 0.9373 | 0.9409 | 0.9384 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0909 | 0.0833 | 0.0870 | | 0.0006 | 12.9172 | 250000 | 0.2495 | 18.2982 | 0.8213 | 0.8236 | 0.8219 | 0.8454 | 0.8480 | 0.8461 | 0.8501 | 0.8533 | 0.8512 | 0.9374 | 0.9428 | 0.9394 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0015 | 13.4339 | 260000 | 0.2507 | 17.9647 | 0.8283 | 0.8315 | 0.8294 | 0.8508 | 0.8541 | 0.8519 | 0.8554 | 0.8589 | 0.8566 | 0.9401 | 0.9460 | 0.9424 | 0.9167 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.0005 | 13.9506 | 270000 | 0.2497 | 18.2668 | 0.8255 | 0.8255 | 0.8250 | 0.8481 | 0.8483 | 0.8477 | 0.8528 | 0.8534 | 0.8526 | 0.9387 | 0.9418 | 0.9396 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0625 | 0.0714 | 0.0667 | | 0.0003 | 14.4673 | 280000 | 0.2543 | 18.2038 | 0.8243 | 0.8289 | 0.8261 | 0.8466 | 0.8515 | 0.8485 | 0.8513 | 0.8567 | 0.8534 | 0.9359 | 0.9427 | 0.9386 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0004 | 14.9840 | 290000 | 0.2506 | 17.8357 | 0.8278 | 0.8286 | 0.8277 | 0.8503 | 0.8513 | 0.8503 | 0.8552 | 0.8568 | 0.8555 | 0.9404 | 0.9445 | 0.9418 | 0.9167 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0909 | 0.0909 | 0.0909 | | 0.0005 | 15.5007 | 300000 | 0.2551 | 17.8923 | 0.8219 | 0.8259 | 0.8234 | 0.8449 | 0.8492 | 0.8465 | 0.8495 | 0.8541 | 0.8513 | 0.9391 | 0.9453 | 0.9415 | 0.9167 | 0.9167 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0003 | 16.0174 | 310000 | 0.2566 | 17.6469 | 0.8287 | 0.8285 | 0.8281 | 0.8517 | 0.8516 | 0.8512 | 0.8567 | 0.8568 | 0.8562 | 0.9431 | 0.9454 | 0.9437 | 0.9231 | 0.9167 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0769 | 0.1 | 0.0870 | | 0.0005 | 16.5340 | 320000 | 0.2564 | 17.8168 | 0.8242 | 0.8279 | 0.8255 | 0.8478 | 0.8516 | 0.8492 | 0.8525 | 0.8564 | 0.8539 | 0.9428 | 0.9477 | 0.9446 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.0003 | 17.0507 | 330000 | 0.2518 | 17.5682 | 0.8289 | 0.8328 | 0.8304 | 0.8521 | 0.8562 | 0.8536 | 0.8565 | 0.8608 | 0.8581 | 0.9439 | 0.9501 | 0.9464 | 0.9167 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0002 | 17.5674 | 340000 | 0.2561 | 17.6311 | 0.8281 | 0.8300 | 0.8286 | 0.8508 | 0.8528 | 0.8513 | 0.8555 | 0.8577 | 0.8561 | 0.9434 | 0.9469 | 0.9445 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0002 | 18.0841 | 350000 | 0.2566 | 17.5399 | 0.8275 | 0.8301 | 0.8283 | 0.8505 | 0.8532 | 0.8513 | 0.8552 | 0.8582 | 0.8562 | 0.9433 | 0.9487 | 0.9453 | 0.9167 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0007 | 18.6008 | 360000 | 0.2592 | 17.6437 | 0.8299 | 0.8336 | 0.8312 | 0.8530 | 0.8568 | 0.8544 | 0.8574 | 0.8616 | 0.8590 | 0.9432 | 0.9487 | 0.9453 | 0.9167 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0003 | 19.1175 | 370000 | 0.2609 | 17.4958 | 0.8337 | 0.8362 | 0.8344 | 0.8570 | 0.8597 | 0.8578 | 0.8615 | 0.8646 | 0.8625 | 0.9450 | 0.9499 | 0.9468 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0 | 19.6342 | 380000 | 0.2601 | 17.5053 | 0.8286 | 0.8305 | 0.8291 | 0.8513 | 0.8534 | 0.8519 | 0.8558 | 0.8582 | 0.8565 | 0.9446 | 0.9486 | 0.9460 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0 | 20.1509 | 390000 | 0.2595 | 16.9200 | 0.8378 | 0.8387 | 0.8378 | 0.8602 | 0.8610 | 0.8601 | 0.8648 | 0.8658 | 0.8648 | 0.9480 | 0.9499 | 0.9484 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.1 | 0.1000 | | 0.0 | 20.6676 | 400000 | 0.2615 | 17.1528 | 0.8337 | 0.8337 | 0.8332 | 0.8558 | 0.8558 | 0.8553 | 0.8603 | 0.8608 | 0.8601 | 0.9464 | 0.9490 | 0.9470 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0 | 21.1843 | 410000 | 0.2617 | 17.0333 | 0.8351 | 0.8345 | 0.8344 | 0.8565 | 0.8559 | 0.8557 | 0.8608 | 0.8605 | 0.8601 | 0.9472 | 0.9484 | 0.9472 | 0.9231 | 0.9231 | 0.9286 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | | 0.0 | 21.7009 | 420000 | 0.2644 | 17.0301 | 0.8360 | 0.8353 | 0.8352 | 0.8586 | 0.8578 | 0.8577 | 0.8635 | 0.8631 | 0.8628 | 0.9484 | 0.9498 | 0.9485 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0002 | 22.2176 | 430000 | 0.2624 | 17.1591 | 0.8329 | 0.8333 | 0.8326 | 0.8550 | 0.8555 | 0.8548 | 0.8599 | 0.8608 | 0.8598 | 0.9470 | 0.9494 | 0.9475 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.0 | 22.7343 | 440000 | 0.2643 | 17.1402 | 0.8330 | 0.8347 | 0.8333 | 0.8549 | 0.8567 | 0.8553 | 0.8597 | 0.8618 | 0.8602 | 0.9455 | 0.9487 | 0.9464 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0 | 23.2510 | 450000 | 0.2634 | 16.8287 | 0.8358 | 0.8366 | 0.8357 | 0.8576 | 0.8584 | 0.8575 | 0.8622 | 0.8634 | 0.8623 | 0.9465 | 0.9492 | 0.9472 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | 0.0714 | 0.0833 | | 0.0 | 23.7677 | 460000 | 0.2643 | 16.8098 | 0.8331 | 0.8343 | 0.8332 | 0.8553 | 0.8567 | 0.8555 | 0.8600 | 0.8618 | 0.8604 | 0.9474 | 0.9507 | 0.9485 | 0.9231 | 0.9231 | 0.9231 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0 | 24.2844 | 470000 | 0.2641 | 16.9294 | 0.8361 | 0.8372 | 0.8362 | 0.8581 | 0.8592 | 0.8581 | 0.8626 | 0.8643 | 0.8629 | 0.9471 | 0.9505 | 0.9482 | 0.9231 | 0.9231 | 0.9286 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0 | 24.8011 | 480000 | 0.2646 | 16.8004 | 0.8338 | 0.8348 | 0.8338 | 0.8553 | 0.8563 | 0.8553 | 0.8600 | 0.8617 | 0.8603 | 0.9465 | 0.9497 | 0.9475 | 0.9231 | 0.9231 | 0.9286 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0 | 25.3178 | 490000 | 0.2639 | 16.7154 | 0.8372 | 0.8387 | 0.8374 | 0.8589 | 0.8605 | 0.8592 | 0.8636 | 0.8658 | 0.8642 | 0.9475 | 0.9509 | 0.9486 | 0.9231 | 0.9231 | 0.9286 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0 | 25.8345 | 500000 | 0.2642 | 16.6210 | 0.8369 | 0.8382 | 0.8371 | 0.8586 | 0.8599 | 0.8588 | 0.8633 | 0.8652 | 0.8637 | 0.9480 | 0.9510 | 0.9488 | 0.9231 | 0.9231 | 0.9286 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | ### Framework versions - Transformers 4.41.2 - Pytorch 2.2.1 - Datasets 2.20.0 - Tokenizers 0.19.1