xlm-roberta-base-trading

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2400
• Accuracy: 0.9603

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	227	0.1720	0.9488
No log	2.0	454	0.1644	0.9427
0.5189	3.0	681	0.1209	0.9565
0.5189	4.0	908	0.1223	0.9557
0.0997	5.0	1135	0.1350	0.9535
0.0997	6.0	1362	0.1239	0.9557
0.0729	7.0	1589	0.1245	0.9581
0.0729	8.0	1816	0.1264	0.9568
0.0578	9.0	2043	0.1313	0.9590
0.0578	10.0	2270	0.1502	0.9540
0.0578	11.0	2497	0.1411	0.9573
0.0489	12.0	2724	0.1527	0.9581
0.0489	13.0	2951	0.1537	0.9562
0.0419	14.0	3178	0.1561	0.9581
0.0419	15.0	3405	0.1686	0.9592
0.0363	16.0	3632	0.1730	0.9559
0.0363	17.0	3859	0.1684	0.9603
0.0337	18.0	4086	0.1764	0.9581
0.0337	19.0	4313	0.1725	0.9592
0.0289	20.0	4540	0.1677	0.9595
0.0289	21.0	4767	0.1726	0.9570
0.0289	22.0	4994	0.1802	0.9614
0.0258	23.0	5221	0.1984	0.9587
0.0258	24.0	5448	0.1915	0.9584
0.0253	25.0	5675	0.2046	0.9587
0.0253	26.0	5902	0.2221	0.9592
0.0231	27.0	6129	0.2321	0.9584
0.0231	28.0	6356	0.2018	0.9562
0.0185	29.0	6583	0.2385	0.9592
0.0185	30.0	6810	0.2219	0.9598
0.0187	31.0	7037	0.2097	0.9609
0.0187	32.0	7264	0.2204	0.9606
0.0187	33.0	7491	0.2174	0.9598
0.0163	34.0	7718	0.2310	0.9601
0.0163	35.0	7945	0.2349	0.9603
0.0147	36.0	8172	0.2426	0.9595
0.0147	37.0	8399	0.2404	0.9592
0.0151	38.0	8626	0.2357	0.9609
0.0151	39.0	8853	0.2390	0.9601
0.0132	40.0	9080	0.2400	0.9603

Framework versions

• Transformers 4.40.2
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.19.1