Edit model card

gemma2b_quotes_v4

This model is a fine-tuned version of google/gemma-2b on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.5350

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_steps: 0.03
  • training_steps: 50

Training results

Training Loss Epoch Step Validation Loss
2.4383 0.0100 5 2.8428
1.8807 0.0199 10 2.6970
2.389 0.0299 15 2.7069
2.2562 0.0399 20 2.6762
2.0517 0.0499 25 2.6469
1.9751 0.0598 30 2.6153
2.2206 0.0698 35 2.6186
2.2526 0.0798 40 2.5952
1.9522 0.0897 45 2.5328
1.8209 0.0997 50 2.5350

Framework versions

  • PEFT 0.11.1
  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
3
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for dwb2023/gemma2b_quotes_v4

Base model

google/gemma-2b
Adapter
(23032)
this model