Hugging Face's logo

ThWu
/
tmp

Text Generation
Transformers
Safetensors
llama
trl
sft
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
tmp / README.md
ThWu's picture
ThWu
Model save
5221720 verified
preview code
|
raw
history blame contribute delete
13.1 kB
metadata
library_name: transformers
license: llama3.2
base_model: meta-llama/Llama-3.2-1B
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: tmp
    results: []

tmp

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.3192

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1.41e-05
  • train_batch_size: 2
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3.0

Training results

Training Loss Epoch Step Validation Loss
1.5915 0.0134 50 1.2903
1.3717 0.0268 100 1.2596
1.0418 0.0401 150 1.2804
1.2548 0.0535 200 1.2606
1.3994 0.0669 250 1.2454
1.1584 0.0803 300 1.2351
1.0075 0.0937 350 1.2278
1.3926 0.1070 400 1.2254
1.2131 0.1204 450 1.2212
1.3407 0.1338 500 1.2100
1.042 0.1472 550 1.2228
1.398 0.1606 600 1.2049
0.9886 0.1739 650 1.2033
1.3415 0.1873 700 1.1979
1.414 0.2007 750 1.1973
1.0634 0.2141 800 1.1925
0.9591 0.2275 850 1.1846
1.4814 0.2408 900 1.1816
1.4658 0.2542 950 1.1804
1.3086 0.2676 1000 1.1789
0.9067 0.2810 1050 1.1678
1.0266 0.2944 1100 1.1679
1.6225 0.3077 1150 1.1694
1.206 0.3211 1200 1.1668
1.2348 0.3345 1250 1.1628
1.3967 0.3479 1300 1.1572
1.3526 0.3613 1350 1.1558
1.0515 0.3746 1400 1.1578
1.2215 0.3880 1450 1.1574
0.8743 0.4014 1500 1.1516
1.5303 0.4148 1550 1.1461
1.1828 0.4282 1600 1.1523
0.9266 0.4415 1650 1.1443
1.3904 0.4549 1700 1.1358
1.138 0.4683 1750 1.1440
1.3723 0.4817 1800 1.1389
1.2073 0.4950 1850 1.1416
1.1665 0.5084 1900 1.1335
1.2742 0.5218 1950 1.1289
1.1677 0.5352 2000 1.1286
1.0681 0.5486 2050 1.1289
0.8086 0.5619 2100 1.1200
0.79 0.5753 2150 1.1245
0.9748 0.5887 2200 1.1275
1.2156 0.6021 2250 1.1204
0.8723 0.6155 2300 1.1151
0.9383 0.6288 2350 1.1160
1.0047 0.6422 2400 1.1169
0.9831 0.6556 2450 1.1192
0.7517 0.6690 2500 1.1098
1.3771 0.6824 2550 1.1128
1.0822 0.6957 2600 1.1158
1.0965 0.7091 2650 1.1073
1.0562 0.7225 2700 1.1108
1.0419 0.7359 2750 1.1184
0.8352 0.7493 2800 1.1060
1.0286 0.7626 2850 1.1043
0.9745 0.7760 2900 1.1019
0.9868 0.7894 2950 1.0965
1.0109 0.8028 3000 1.0978
1.437 0.8162 3050 1.0969
0.8 0.8295 3100 1.0882
1.1526 0.8429 3150 1.0912
1.052 0.8563 3200 1.0922
1.1689 0.8697 3250 1.0871
1.3413 0.8831 3300 1.0851
1.1188 0.8964 3350 1.0833
1.625 0.9098 3400 1.0867
1.3762 0.9232 3450 1.0816
1.0802 0.9366 3500 1.0825
0.9063 0.9500 3550 1.0767
1.0199 0.9633 3600 1.0783
1.5628 0.9767 3650 1.0750
1.0558 0.9901 3700 1.0774
0.7092 1.0035 3750 1.0841
0.7194 1.0169 3800 1.1159
0.8033 1.0302 3850 1.1189
0.5744 1.0436 3900 1.1321
0.6601 1.0570 3950 1.1199
0.8371 1.0704 4000 1.1241
0.8107 1.0838 4050 1.1225
0.6045 1.0971 4100 1.1291
0.6476 1.1105 4150 1.1280
0.6125 1.1239 4200 1.1228
0.5005 1.1373 4250 1.1239
0.7029 1.1507 4300 1.1302
0.7131 1.1640 4350 1.1217
0.7028 1.1774 4400 1.1266
0.7679 1.1908 4450 1.1164
0.7504 1.2042 4500 1.1235
0.7788 1.2176 4550 1.1253
0.6972 1.2309 4600 1.1166
1.0489 1.2443 4650 1.1204
0.4751 1.2577 4700 1.1185
0.5464 1.2711 4750 1.1254
0.7255 1.2845 4800 1.1202
0.8914 1.2978 4850 1.1193
0.5107 1.3112 4900 1.1252
0.8114 1.3246 4950 1.1243
0.6298 1.3380 5000 1.1261
0.9236 1.3514 5050 1.1245
0.7085 1.3647 5100 1.1213
0.7505 1.3781 5150 1.1127
0.7309 1.3915 5200 1.1178
0.5225 1.4049 5250 1.1216
0.8705 1.4182 5300 1.1134
0.5532 1.4316 5350 1.1193
0.4079 1.4450 5400 1.1142
0.5628 1.4584 5450 1.1138
0.716 1.4718 5500 1.1126
0.382 1.4851 5550 1.1150
0.6474 1.4985 5600 1.1143
0.6119 1.5119 5650 1.1112
0.4815 1.5253 5700 1.1047
0.8477 1.5387 5750 1.1158
0.8981 1.5520 5800 1.1108
0.639 1.5654 5850 1.1141
0.727 1.5788 5900 1.1137
0.8175 1.5922 5950 1.1116
0.7431 1.6056 6000 1.1152
0.6324 1.6189 6050 1.1145
1.0941 1.6323 6100 1.1142
0.6437 1.6457 6150 1.1082
0.5857 1.6591 6200 1.1103
0.4056 1.6725 6250 1.1137
0.6483 1.6858 6300 1.1069
0.6741 1.6992 6350 1.1027
0.7587 1.7126 6400 1.1087
0.7206 1.7260 6450 1.1156
0.451 1.7394 6500 1.1074
0.8237 1.7527 6550 1.1055
0.6333 1.7661 6600 1.1078
0.6317 1.7795 6650 1.1049
0.6688 1.7929 6700 1.1011
0.6598 1.8063 6750 1.1030
0.642 1.8196 6800 1.1059
0.587 1.8330 6850 1.1002
0.7726 1.8464 6900 1.0966
0.8227 1.8598 6950 1.1014
0.9093 1.8732 7000 1.1011
0.6117 1.8865 7050 1.0999
0.8338 1.8999 7100 1.0937
0.7215 1.9133 7150 1.0935
0.6242 1.9267 7200 1.0909
0.571 1.9401 7250 1.0990
0.7773 1.9534 7300 1.0955
0.7082 1.9668 7350 1.0955
0.7165 1.9802 7400 1.0982
0.5604 1.9936 7450 1.0985
0.3232 2.0070 7500 1.1841
0.3628 2.0203 7550 1.2569
0.4465 2.0337 7600 1.2687
0.3233 2.0471 7650 1.2720
0.281 2.0605 7700 1.2859
0.2199 2.0739 7750 1.2808
0.4787 2.0872 7800 1.2839
0.4288 2.1006 7850 1.2918
0.2966 2.1140 7900 1.3063
0.4248 2.1274 7950 1.3061
0.2717 2.1408 8000 1.2926
0.3561 2.1541 8050 1.3054
0.3736 2.1675 8100 1.2947
0.2936 2.1809 8150 1.3021
0.3316 2.1943 8200 1.2981
0.2931 2.2077 8250 1.3007
0.4591 2.2210 8300 1.2972
0.3023 2.2344 8350 1.3127
0.3407 2.2478 8400 1.3110
0.2361 2.2612 8450 1.3071
0.3509 2.2746 8500 1.3021
0.3868 2.2879 8550 1.3168
0.3218 2.3013 8600 1.3156
0.2913 2.3147 8650 1.3034
0.437 2.3281 8700 1.3214
0.4314 2.3415 8750 1.3136
0.3151 2.3548 8800 1.3085
0.3236 2.3682 8850 1.3100
0.3416 2.3816 8900 1.3050
0.3333 2.3950 8950 1.3151
0.2742 2.4083 9000 1.3153
0.3143 2.4217 9050 1.3243
0.4152 2.4351 9100 1.3164
0.219 2.4485 9150 1.3233
0.4057 2.4619 9200 1.3073
0.3571 2.4752 9250 1.3084
0.3163 2.4886 9300 1.3184
0.3185 2.5020 9350 1.3092
0.4474 2.5154 9400 1.3185
0.1927 2.5288 9450 1.3158
0.2362 2.5421 9500 1.3093
0.3651 2.5555 9550 1.3116
0.2531 2.5689 9600 1.3121
0.2219 2.5823 9650 1.3192
0.2546 2.5957 9700 1.3170
0.2841 2.6090 9750 1.3180
0.3039 2.6224 9800 1.3188
0.3866 2.6358 9850 1.3253
0.378 2.6492 9900 1.3143
0.2671 2.6626 9950 1.3143
0.2715 2.6759 10000 1.3220
0.2104 2.6893 10050 1.3275
0.2663 2.7027 10100 1.3186
0.3433 2.7161 10150 1.3201
0.3493 2.7295 10200 1.3169
0.3615 2.7428 10250 1.3184
0.2843 2.7562 10300 1.3196
0.263 2.7696 10350 1.3158
0.2971 2.7830 10400 1.3136
0.2198 2.7964 10450 1.3231
0.1814 2.8097 10500 1.3187
0.303 2.8231 10550 1.3175
0.4044 2.8365 10600 1.3171
0.2374 2.8499 10650 1.3212
0.2155 2.8633 10700 1.3229
0.2656 2.8766 10750 1.3251
0.2552 2.8900 10800 1.3184
0.2838 2.9034 10850 1.3198
0.2824 2.9168 10900 1.3192
0.2748 2.9302 10950 1.3172
0.2951 2.9435 11000 1.3193
0.3339 2.9569 11050 1.3196
0.3167 2.9703 11100 1.3195
0.2751 2.9837 11150 1.3192
0.3687 2.9971 11200 1.3192

Framework versions

  • Transformers 4.45.2
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.20.1