|
--- |
|
library_name: transformers |
|
license: llama3.2 |
|
base_model: meta-llama/Llama-3.2-1B |
|
tags: |
|
- trl |
|
- sft |
|
- generated_from_trainer |
|
model-index: |
|
- name: tmp |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# tmp |
|
|
|
This model is a fine-tuned version of [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.3192 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1.41e-05 |
|
- train_batch_size: 2 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 3.0 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:------:|:-----:|:---------------:| |
|
| 1.5915 | 0.0134 | 50 | 1.2903 | |
|
| 1.3717 | 0.0268 | 100 | 1.2596 | |
|
| 1.0418 | 0.0401 | 150 | 1.2804 | |
|
| 1.2548 | 0.0535 | 200 | 1.2606 | |
|
| 1.3994 | 0.0669 | 250 | 1.2454 | |
|
| 1.1584 | 0.0803 | 300 | 1.2351 | |
|
| 1.0075 | 0.0937 | 350 | 1.2278 | |
|
| 1.3926 | 0.1070 | 400 | 1.2254 | |
|
| 1.2131 | 0.1204 | 450 | 1.2212 | |
|
| 1.3407 | 0.1338 | 500 | 1.2100 | |
|
| 1.042 | 0.1472 | 550 | 1.2228 | |
|
| 1.398 | 0.1606 | 600 | 1.2049 | |
|
| 0.9886 | 0.1739 | 650 | 1.2033 | |
|
| 1.3415 | 0.1873 | 700 | 1.1979 | |
|
| 1.414 | 0.2007 | 750 | 1.1973 | |
|
| 1.0634 | 0.2141 | 800 | 1.1925 | |
|
| 0.9591 | 0.2275 | 850 | 1.1846 | |
|
| 1.4814 | 0.2408 | 900 | 1.1816 | |
|
| 1.4658 | 0.2542 | 950 | 1.1804 | |
|
| 1.3086 | 0.2676 | 1000 | 1.1789 | |
|
| 0.9067 | 0.2810 | 1050 | 1.1678 | |
|
| 1.0266 | 0.2944 | 1100 | 1.1679 | |
|
| 1.6225 | 0.3077 | 1150 | 1.1694 | |
|
| 1.206 | 0.3211 | 1200 | 1.1668 | |
|
| 1.2348 | 0.3345 | 1250 | 1.1628 | |
|
| 1.3967 | 0.3479 | 1300 | 1.1572 | |
|
| 1.3526 | 0.3613 | 1350 | 1.1558 | |
|
| 1.0515 | 0.3746 | 1400 | 1.1578 | |
|
| 1.2215 | 0.3880 | 1450 | 1.1574 | |
|
| 0.8743 | 0.4014 | 1500 | 1.1516 | |
|
| 1.5303 | 0.4148 | 1550 | 1.1461 | |
|
| 1.1828 | 0.4282 | 1600 | 1.1523 | |
|
| 0.9266 | 0.4415 | 1650 | 1.1443 | |
|
| 1.3904 | 0.4549 | 1700 | 1.1358 | |
|
| 1.138 | 0.4683 | 1750 | 1.1440 | |
|
| 1.3723 | 0.4817 | 1800 | 1.1389 | |
|
| 1.2073 | 0.4950 | 1850 | 1.1416 | |
|
| 1.1665 | 0.5084 | 1900 | 1.1335 | |
|
| 1.2742 | 0.5218 | 1950 | 1.1289 | |
|
| 1.1677 | 0.5352 | 2000 | 1.1286 | |
|
| 1.0681 | 0.5486 | 2050 | 1.1289 | |
|
| 0.8086 | 0.5619 | 2100 | 1.1200 | |
|
| 0.79 | 0.5753 | 2150 | 1.1245 | |
|
| 0.9748 | 0.5887 | 2200 | 1.1275 | |
|
| 1.2156 | 0.6021 | 2250 | 1.1204 | |
|
| 0.8723 | 0.6155 | 2300 | 1.1151 | |
|
| 0.9383 | 0.6288 | 2350 | 1.1160 | |
|
| 1.0047 | 0.6422 | 2400 | 1.1169 | |
|
| 0.9831 | 0.6556 | 2450 | 1.1192 | |
|
| 0.7517 | 0.6690 | 2500 | 1.1098 | |
|
| 1.3771 | 0.6824 | 2550 | 1.1128 | |
|
| 1.0822 | 0.6957 | 2600 | 1.1158 | |
|
| 1.0965 | 0.7091 | 2650 | 1.1073 | |
|
| 1.0562 | 0.7225 | 2700 | 1.1108 | |
|
| 1.0419 | 0.7359 | 2750 | 1.1184 | |
|
| 0.8352 | 0.7493 | 2800 | 1.1060 | |
|
| 1.0286 | 0.7626 | 2850 | 1.1043 | |
|
| 0.9745 | 0.7760 | 2900 | 1.1019 | |
|
| 0.9868 | 0.7894 | 2950 | 1.0965 | |
|
| 1.0109 | 0.8028 | 3000 | 1.0978 | |
|
| 1.437 | 0.8162 | 3050 | 1.0969 | |
|
| 0.8 | 0.8295 | 3100 | 1.0882 | |
|
| 1.1526 | 0.8429 | 3150 | 1.0912 | |
|
| 1.052 | 0.8563 | 3200 | 1.0922 | |
|
| 1.1689 | 0.8697 | 3250 | 1.0871 | |
|
| 1.3413 | 0.8831 | 3300 | 1.0851 | |
|
| 1.1188 | 0.8964 | 3350 | 1.0833 | |
|
| 1.625 | 0.9098 | 3400 | 1.0867 | |
|
| 1.3762 | 0.9232 | 3450 | 1.0816 | |
|
| 1.0802 | 0.9366 | 3500 | 1.0825 | |
|
| 0.9063 | 0.9500 | 3550 | 1.0767 | |
|
| 1.0199 | 0.9633 | 3600 | 1.0783 | |
|
| 1.5628 | 0.9767 | 3650 | 1.0750 | |
|
| 1.0558 | 0.9901 | 3700 | 1.0774 | |
|
| 0.7092 | 1.0035 | 3750 | 1.0841 | |
|
| 0.7194 | 1.0169 | 3800 | 1.1159 | |
|
| 0.8033 | 1.0302 | 3850 | 1.1189 | |
|
| 0.5744 | 1.0436 | 3900 | 1.1321 | |
|
| 0.6601 | 1.0570 | 3950 | 1.1199 | |
|
| 0.8371 | 1.0704 | 4000 | 1.1241 | |
|
| 0.8107 | 1.0838 | 4050 | 1.1225 | |
|
| 0.6045 | 1.0971 | 4100 | 1.1291 | |
|
| 0.6476 | 1.1105 | 4150 | 1.1280 | |
|
| 0.6125 | 1.1239 | 4200 | 1.1228 | |
|
| 0.5005 | 1.1373 | 4250 | 1.1239 | |
|
| 0.7029 | 1.1507 | 4300 | 1.1302 | |
|
| 0.7131 | 1.1640 | 4350 | 1.1217 | |
|
| 0.7028 | 1.1774 | 4400 | 1.1266 | |
|
| 0.7679 | 1.1908 | 4450 | 1.1164 | |
|
| 0.7504 | 1.2042 | 4500 | 1.1235 | |
|
| 0.7788 | 1.2176 | 4550 | 1.1253 | |
|
| 0.6972 | 1.2309 | 4600 | 1.1166 | |
|
| 1.0489 | 1.2443 | 4650 | 1.1204 | |
|
| 0.4751 | 1.2577 | 4700 | 1.1185 | |
|
| 0.5464 | 1.2711 | 4750 | 1.1254 | |
|
| 0.7255 | 1.2845 | 4800 | 1.1202 | |
|
| 0.8914 | 1.2978 | 4850 | 1.1193 | |
|
| 0.5107 | 1.3112 | 4900 | 1.1252 | |
|
| 0.8114 | 1.3246 | 4950 | 1.1243 | |
|
| 0.6298 | 1.3380 | 5000 | 1.1261 | |
|
| 0.9236 | 1.3514 | 5050 | 1.1245 | |
|
| 0.7085 | 1.3647 | 5100 | 1.1213 | |
|
| 0.7505 | 1.3781 | 5150 | 1.1127 | |
|
| 0.7309 | 1.3915 | 5200 | 1.1178 | |
|
| 0.5225 | 1.4049 | 5250 | 1.1216 | |
|
| 0.8705 | 1.4182 | 5300 | 1.1134 | |
|
| 0.5532 | 1.4316 | 5350 | 1.1193 | |
|
| 0.4079 | 1.4450 | 5400 | 1.1142 | |
|
| 0.5628 | 1.4584 | 5450 | 1.1138 | |
|
| 0.716 | 1.4718 | 5500 | 1.1126 | |
|
| 0.382 | 1.4851 | 5550 | 1.1150 | |
|
| 0.6474 | 1.4985 | 5600 | 1.1143 | |
|
| 0.6119 | 1.5119 | 5650 | 1.1112 | |
|
| 0.4815 | 1.5253 | 5700 | 1.1047 | |
|
| 0.8477 | 1.5387 | 5750 | 1.1158 | |
|
| 0.8981 | 1.5520 | 5800 | 1.1108 | |
|
| 0.639 | 1.5654 | 5850 | 1.1141 | |
|
| 0.727 | 1.5788 | 5900 | 1.1137 | |
|
| 0.8175 | 1.5922 | 5950 | 1.1116 | |
|
| 0.7431 | 1.6056 | 6000 | 1.1152 | |
|
| 0.6324 | 1.6189 | 6050 | 1.1145 | |
|
| 1.0941 | 1.6323 | 6100 | 1.1142 | |
|
| 0.6437 | 1.6457 | 6150 | 1.1082 | |
|
| 0.5857 | 1.6591 | 6200 | 1.1103 | |
|
| 0.4056 | 1.6725 | 6250 | 1.1137 | |
|
| 0.6483 | 1.6858 | 6300 | 1.1069 | |
|
| 0.6741 | 1.6992 | 6350 | 1.1027 | |
|
| 0.7587 | 1.7126 | 6400 | 1.1087 | |
|
| 0.7206 | 1.7260 | 6450 | 1.1156 | |
|
| 0.451 | 1.7394 | 6500 | 1.1074 | |
|
| 0.8237 | 1.7527 | 6550 | 1.1055 | |
|
| 0.6333 | 1.7661 | 6600 | 1.1078 | |
|
| 0.6317 | 1.7795 | 6650 | 1.1049 | |
|
| 0.6688 | 1.7929 | 6700 | 1.1011 | |
|
| 0.6598 | 1.8063 | 6750 | 1.1030 | |
|
| 0.642 | 1.8196 | 6800 | 1.1059 | |
|
| 0.587 | 1.8330 | 6850 | 1.1002 | |
|
| 0.7726 | 1.8464 | 6900 | 1.0966 | |
|
| 0.8227 | 1.8598 | 6950 | 1.1014 | |
|
| 0.9093 | 1.8732 | 7000 | 1.1011 | |
|
| 0.6117 | 1.8865 | 7050 | 1.0999 | |
|
| 0.8338 | 1.8999 | 7100 | 1.0937 | |
|
| 0.7215 | 1.9133 | 7150 | 1.0935 | |
|
| 0.6242 | 1.9267 | 7200 | 1.0909 | |
|
| 0.571 | 1.9401 | 7250 | 1.0990 | |
|
| 0.7773 | 1.9534 | 7300 | 1.0955 | |
|
| 0.7082 | 1.9668 | 7350 | 1.0955 | |
|
| 0.7165 | 1.9802 | 7400 | 1.0982 | |
|
| 0.5604 | 1.9936 | 7450 | 1.0985 | |
|
| 0.3232 | 2.0070 | 7500 | 1.1841 | |
|
| 0.3628 | 2.0203 | 7550 | 1.2569 | |
|
| 0.4465 | 2.0337 | 7600 | 1.2687 | |
|
| 0.3233 | 2.0471 | 7650 | 1.2720 | |
|
| 0.281 | 2.0605 | 7700 | 1.2859 | |
|
| 0.2199 | 2.0739 | 7750 | 1.2808 | |
|
| 0.4787 | 2.0872 | 7800 | 1.2839 | |
|
| 0.4288 | 2.1006 | 7850 | 1.2918 | |
|
| 0.2966 | 2.1140 | 7900 | 1.3063 | |
|
| 0.4248 | 2.1274 | 7950 | 1.3061 | |
|
| 0.2717 | 2.1408 | 8000 | 1.2926 | |
|
| 0.3561 | 2.1541 | 8050 | 1.3054 | |
|
| 0.3736 | 2.1675 | 8100 | 1.2947 | |
|
| 0.2936 | 2.1809 | 8150 | 1.3021 | |
|
| 0.3316 | 2.1943 | 8200 | 1.2981 | |
|
| 0.2931 | 2.2077 | 8250 | 1.3007 | |
|
| 0.4591 | 2.2210 | 8300 | 1.2972 | |
|
| 0.3023 | 2.2344 | 8350 | 1.3127 | |
|
| 0.3407 | 2.2478 | 8400 | 1.3110 | |
|
| 0.2361 | 2.2612 | 8450 | 1.3071 | |
|
| 0.3509 | 2.2746 | 8500 | 1.3021 | |
|
| 0.3868 | 2.2879 | 8550 | 1.3168 | |
|
| 0.3218 | 2.3013 | 8600 | 1.3156 | |
|
| 0.2913 | 2.3147 | 8650 | 1.3034 | |
|
| 0.437 | 2.3281 | 8700 | 1.3214 | |
|
| 0.4314 | 2.3415 | 8750 | 1.3136 | |
|
| 0.3151 | 2.3548 | 8800 | 1.3085 | |
|
| 0.3236 | 2.3682 | 8850 | 1.3100 | |
|
| 0.3416 | 2.3816 | 8900 | 1.3050 | |
|
| 0.3333 | 2.3950 | 8950 | 1.3151 | |
|
| 0.2742 | 2.4083 | 9000 | 1.3153 | |
|
| 0.3143 | 2.4217 | 9050 | 1.3243 | |
|
| 0.4152 | 2.4351 | 9100 | 1.3164 | |
|
| 0.219 | 2.4485 | 9150 | 1.3233 | |
|
| 0.4057 | 2.4619 | 9200 | 1.3073 | |
|
| 0.3571 | 2.4752 | 9250 | 1.3084 | |
|
| 0.3163 | 2.4886 | 9300 | 1.3184 | |
|
| 0.3185 | 2.5020 | 9350 | 1.3092 | |
|
| 0.4474 | 2.5154 | 9400 | 1.3185 | |
|
| 0.1927 | 2.5288 | 9450 | 1.3158 | |
|
| 0.2362 | 2.5421 | 9500 | 1.3093 | |
|
| 0.3651 | 2.5555 | 9550 | 1.3116 | |
|
| 0.2531 | 2.5689 | 9600 | 1.3121 | |
|
| 0.2219 | 2.5823 | 9650 | 1.3192 | |
|
| 0.2546 | 2.5957 | 9700 | 1.3170 | |
|
| 0.2841 | 2.6090 | 9750 | 1.3180 | |
|
| 0.3039 | 2.6224 | 9800 | 1.3188 | |
|
| 0.3866 | 2.6358 | 9850 | 1.3253 | |
|
| 0.378 | 2.6492 | 9900 | 1.3143 | |
|
| 0.2671 | 2.6626 | 9950 | 1.3143 | |
|
| 0.2715 | 2.6759 | 10000 | 1.3220 | |
|
| 0.2104 | 2.6893 | 10050 | 1.3275 | |
|
| 0.2663 | 2.7027 | 10100 | 1.3186 | |
|
| 0.3433 | 2.7161 | 10150 | 1.3201 | |
|
| 0.3493 | 2.7295 | 10200 | 1.3169 | |
|
| 0.3615 | 2.7428 | 10250 | 1.3184 | |
|
| 0.2843 | 2.7562 | 10300 | 1.3196 | |
|
| 0.263 | 2.7696 | 10350 | 1.3158 | |
|
| 0.2971 | 2.7830 | 10400 | 1.3136 | |
|
| 0.2198 | 2.7964 | 10450 | 1.3231 | |
|
| 0.1814 | 2.8097 | 10500 | 1.3187 | |
|
| 0.303 | 2.8231 | 10550 | 1.3175 | |
|
| 0.4044 | 2.8365 | 10600 | 1.3171 | |
|
| 0.2374 | 2.8499 | 10650 | 1.3212 | |
|
| 0.2155 | 2.8633 | 10700 | 1.3229 | |
|
| 0.2656 | 2.8766 | 10750 | 1.3251 | |
|
| 0.2552 | 2.8900 | 10800 | 1.3184 | |
|
| 0.2838 | 2.9034 | 10850 | 1.3198 | |
|
| 0.2824 | 2.9168 | 10900 | 1.3192 | |
|
| 0.2748 | 2.9302 | 10950 | 1.3172 | |
|
| 0.2951 | 2.9435 | 11000 | 1.3193 | |
|
| 0.3339 | 2.9569 | 11050 | 1.3196 | |
|
| 0.3167 | 2.9703 | 11100 | 1.3195 | |
|
| 0.2751 | 2.9837 | 11150 | 1.3192 | |
|
| 0.3687 | 2.9971 | 11200 | 1.3192 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.45.2 |
|
- Pytorch 2.4.1+cu121 |
|
- Datasets 3.0.1 |
|
- Tokenizers 0.20.1 |
|
|
