AraT5v2-base-1024-p-l-akk-en-20240801-201225

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0402

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss
0.0561	0.0552	2500	0.0493
0.0511	0.1105	5000	0.0501
0.0475	0.1657	7500	0.0499
0.0465	0.2210	10000	0.0499
0.0417	0.2762	12500	0.0495
0.0528	0.3314	15000	0.0497
0.0523	0.3867	17500	0.0492
0.0492	0.4419	20000	0.0497
0.0468	0.4972	22500	0.0488
0.0515	0.5524	25000	0.0489
0.0522	0.6076	27500	0.0487
0.0522	0.6629	30000	0.0489
0.0544	0.7181	32500	0.0486
0.0466	0.7734	35000	0.0488
0.0546	0.8286	37500	0.0491
0.0558	0.8838	40000	0.0486
0.0532	0.9391	42500	0.0484
0.049	0.9943	45000	0.0484
0.049	1.0496	47500	0.0487
0.0429	1.1048	50000	0.0483
0.0549	1.1600	52500	0.0482
0.047	1.2153	55000	0.0480
0.0446	1.2705	57500	0.0477
0.0583	1.3258	60000	0.0478
0.0471	1.3810	62500	0.0477
0.0549	1.4362	65000	0.0475
0.0441	1.4915	67500	0.0476
0.0467	1.5467	70000	0.0471
0.0434	1.6020	72500	0.0467
0.0522	1.6572	75000	0.0471
0.0513	1.7124	77500	0.0469
0.0492	1.7677	80000	0.0465
0.0428	1.8229	82500	0.0466
0.0479	1.8782	85000	0.0461
0.0522	1.9334	87500	0.0463
0.0503	1.9886	90000	0.0462
0.0378	2.0439	92500	0.0463
0.0442	2.0991	95000	0.0462
0.0412	2.1544	97500	0.0461
0.0378	2.2096	100000	0.0460
0.0437	2.2648	102500	0.0460
0.0467	2.3201	105000	0.0456
0.0446	2.3753	107500	0.0455
0.0395	2.4306	110000	0.0456
0.0425	2.4858	112500	0.0453
0.0435	2.5410	115000	0.0454
0.0457	2.5963	117500	0.0454
0.048	2.6515	120000	0.0452
0.0474	2.7068	122500	0.0450
0.0512	2.7620	125000	0.0451
0.0472	2.8172	127500	0.0445
0.0513	2.8725	130000	0.0448
0.0451	2.9277	132500	0.0446
0.0445	2.9830	135000	0.0445
0.0445	3.0382	137500	0.0447
0.0396	3.0934	140000	0.0444
0.047	3.1487	142500	0.0443
0.0422	3.2039	145000	0.0444
0.0443	3.2592	147500	0.0444
0.0483	3.3144	150000	0.0442
0.0434	3.3696	152500	0.0442
0.05	3.4249	155000	0.0441
0.0403	3.4801	157500	0.0438
0.0461	3.5354	160000	0.0437
0.0433	3.5906	162500	0.0437
0.0418	3.6458	165000	0.0437
0.0407	3.7011	167500	0.0436
0.0408	3.7563	170000	0.0435
0.0432	3.8116	172500	0.0434
0.0411	3.8668	175000	0.0434
0.045	3.9220	177500	0.0431
0.0408	3.9773	180000	0.0430
0.0413	4.0325	182500	0.0435
0.0443	4.0878	185000	0.0430
0.0375	4.1430	187500	0.0434
0.0431	4.1982	190000	0.0428
0.0381	4.2535	192500	0.0428
0.0456	4.3087	195000	0.0429
0.0468	4.3640	197500	0.0432
0.0404	4.4192	200000	0.0426
0.0417	4.4744	202500	0.0424
0.0402	4.5297	205000	0.0429
0.0427	4.5849	207500	0.0427
0.0413	4.6402	210000	0.0427
0.0417	4.6954	212500	0.0425
0.0353	4.7506	215000	0.0425
0.039	4.8059	217500	0.0423
0.0427	4.8611	220000	0.0420
0.0359	4.9164	222500	0.0422
0.0392	4.9716	225000	0.0421
0.0424	5.0268	227500	0.0426
0.0398	5.0821	230000	0.0421
0.0398	5.1373	232500	0.0421
0.0349	5.1926	235000	0.0422
0.0415	5.2478	237500	0.0422
0.0413	5.3030	240000	0.0421
0.0422	5.3583	242500	0.0419
0.0365	5.4135	245000	0.0418
0.0403	5.4688	247500	0.0416
0.0398	5.5240	250000	0.0417
0.0393	5.5792	252500	0.0417
0.0396	5.6345	255000	0.0417
0.0383	5.6897	257500	0.0417
0.041	5.7450	260000	0.0417
0.0425	5.8002	262500	0.0414
0.0417	5.8554	265000	0.0412
0.0352	5.9107	267500	0.0413
0.0349	5.9659	270000	0.0413
0.0397	6.0212	272500	0.0413
0.037	6.0764	275000	0.0414
0.0358	6.1316	277500	0.0416
0.0402	6.1869	280000	0.0415
0.0332	6.2421	282500	0.0417
0.035	6.2974	285000	0.0415
0.0364	6.3526	287500	0.0413
0.0427	6.4078	290000	0.0412
0.0387	6.4631	292500	0.0409
0.0288	6.5183	295000	0.0410
0.0417	6.5736	297500	0.0410
0.0372	6.6288	300000	0.0410
0.042	6.6840	302500	0.0411
0.0347	6.7393	305000	0.0409
0.0363	6.7945	307500	0.0408
0.0413	6.8498	310000	0.0410
0.0386	6.9050	312500	0.0407
0.0362	6.9602	315000	0.0407
0.0385	7.0155	317500	0.0410
0.0412	7.0707	320000	0.0410
0.0349	7.1260	322500	0.0407
0.0383	7.1812	325000	0.0408
0.0316	7.2364	327500	0.0410
0.0387	7.2917	330000	0.0409
0.0321	7.3469	332500	0.0407
0.0323	7.4022	335000	0.0406
0.0352	7.4574	337500	0.0406
0.0342	7.5126	340000	0.0404
0.0356	7.5679	342500	0.0406
0.0379	7.6231	345000	0.0405
0.0381	7.6784	347500	0.0406
0.0359	7.7336	350000	0.0403
0.0371	7.7889	352500	0.0403
0.0385	7.8441	355000	0.0404
0.0321	7.8993	357500	0.0405
0.0366	7.9546	360000	0.0406
0.0316	8.0098	362500	0.0407
0.0372	8.0651	365000	0.0406
0.0327	8.1203	367500	0.0406
0.0334	8.1755	370000	0.0405
0.0366	8.2308	372500	0.0405
0.0346	8.2860	375000	0.0405
0.0322	8.3413	377500	0.0404
0.0429	8.3965	380000	0.0403
0.0341	8.4517	382500	0.0404
0.0345	8.5070	385000	0.0403
0.0356	8.5622	387500	0.0403
0.0352	8.6175	390000	0.0404
0.0361	8.6727	392500	0.0402
0.0335	8.7279	395000	0.0404
0.0361	8.7832	397500	0.0403
0.035	8.8384	400000	0.0403
0.0327	8.8937	402500	0.0402
0.0336	8.9489	405000	0.0401
0.035	9.0041	407500	0.0404
0.0363	9.0594	410000	0.0403
0.0317	9.1146	412500	0.0403
0.033	9.1699	415000	0.0404
0.0368	9.2251	417500	0.0403
0.0333	9.2803	420000	0.0403
0.0308	9.3356	422500	0.0402
0.0329	9.3908	425000	0.0403
0.0371	9.4461	427500	0.0402
0.0331	9.5013	430000	0.0403
0.0328	9.5565	432500	0.0402
0.0323	9.6118	435000	0.0402
0.0283	9.6670	437500	0.0402
0.0338	9.7223	440000	0.0401
0.0323	9.7775	442500	0.0402
0.0374	9.8327	445000	0.0402
0.0357	9.8880	447500	0.0402
0.037	9.9432	450000	0.0402
0.0348	9.9985	452500	0.0402

Framework versions

• Transformers 4.44.0.dev0
• Pytorch 2.5.0.dev20240625
• Datasets 2.20.0
• Tokenizers 0.19.1