Edit model card

led-risalah_data_v2

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7850
  • Rouge1 Precision: 0.816
  • Rouge1 Recall: 0.2149
  • Rouge1 Fmeasure: 0.3393

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 8
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Fmeasure Rouge1 Precision Rouge1 Recall
2.4163 0.9143 8 1.9482 0.2001 0.4982 0.1254
1.6578 1.9429 17 1.8076 0.2489 0.6295 0.1554
1.656 2.9143 24 1.4664 0.2459 0.6118 0.154
1.5142 3.9429 33 1.4191 0.2546 0.646 0.159
1.4169 4.9714 42 1.4162 0.27 0.6675 0.1698
1.4123 6.9143 56 1.3197 0.2807 0.7054 0.1755
1.3398 7.9429 65 1.3156 0.2797 0.6912 0.1759
1.146 8.9714 74 1.3247 0.2925 0.728 0.1834
1.1481 10.0 83 1.3366 0.2739 0.6799 0.1718
1.2033 10.9143 91 1.3387 0.2789 0.69 0.1752
1.0855 11.9429 100 1.3375 0.2888 0.7146 0.1814
0.999 12.9714 109 1.3589 0.2922 0.7265 0.1831
1.0034 14.0 118 1.3601 0.2872 0.7157 0.1801
0.9831 14.9143 126 1.3762 0.2851 0.7024 0.1792
0.9347 15.9429 135 1.3743 0.2769 0.6841 0.174
0.9018 16.9714 144 1.3820 0.2862 0.7139 0.1797
0.8939 18.0 153 1.3841 0.2879 0.7134 0.1806

Framework versions

  • Transformers 4.42.3
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
2
Safetensors
Model size
162M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.