jamesliu23's picture
jamesliu23/openhermes-mistral-gptq-dpo
29cc232
metadata
license: apache-2.0
base_model: TheBloke/OpenHermes-2-Mistral-7B-GPTQ
tags:
  - generated_from_trainer
model-index:
  - name: openhermes-mistral-gptq-dpo
    results: []

openhermes-mistral-gptq-dpo

This model is a fine-tuned version of TheBloke/OpenHermes-2-Mistral-7B-GPTQ on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8689
  • Rewards/chosen: -1.0765
  • Rewards/rejected: -0.9664
  • Rewards/accuracies: 0.4800
  • Rewards/margins: -0.1101
  • Logps/rejected: -139.8855
  • Logps/chosen: -135.5758
  • Logits/rejected: -2.2110
  • Logits/chosen: -2.2313

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 2
  • training_steps: 50

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.6842 0.01 10 0.7768 -0.4985 -0.4411 0.5750 -0.0574 -134.6327 -129.7961 -2.2478 -2.2667
0.8539 0.02 20 2.0287 -9.4492 -8.7234 0.4800 -0.7258 -217.4559 -219.3030 -1.5781 -1.5859
0.9433 0.03 30 1.6915 -7.3191 -6.8845 0.5 -0.4346 -199.0667 -198.0014 -1.6865 -1.6973
0.6295 0.04 40 0.9729 -1.6986 -1.5214 0.5200 -0.1772 -145.4362 -141.7968 -2.1859 -2.2057
0.8681 0.05 50 0.8689 -1.0765 -0.9664 0.4800 -0.1101 -139.8855 -135.5758 -2.2110 -2.2313

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.1.0+cu121
  • Datasets 2.14.6
  • Tokenizers 0.14.1