iulik-pisik/vreme_horoscop_small

Vreme Model Small - finetuned on horoscope

This model is a fine-tuned version of iulik-udrik/vreme_model_small on the Horoscop Neti Sandu dataset. It achieves the following results on the evaluation set:

• Loss: 0.2529
• Wer: 10.1728

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.0047	9.71	1000	0.2062	10.5865
0.0004	19.42	2000	0.2349	10.4405
0.0002	29.13	3000	0.2488	10.2701
0.0002	38.83	4000	0.2529	10.1728

Framework versions

• Transformers 4.39.1
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2