southmost/phi2-lora-distilabel-intel-orca-dpo-pairs-init

README.md

@@ -18,15 +18,15 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4547
-- Rewards/chosen: -0.0932
-- Rewards/rejected: -1.3103
-- Rewards/accuracies: 0.8386
-- Rewards/margins: 1.2171
-- Logps/rejected: -222.2418
-- Logps/chosen: -199.7473
-- Logits/rejected: 0.5130
-- Logits/chosen: 0.3441
+- Loss: 0.4467
+- Rewards/chosen: -0.0981
+- Rewards/rejected: -1.3106
+- Rewards/accuracies: 0.8410
+- Rewards/margins: 1.2125
+- Logps/rejected: -228.4777
+- Logps/chosen: -209.0628
+- Logits/rejected: 0.4528
+- Logits/chosen: 0.2946
 
 ## Model description
 
@@ -60,13 +60,13 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
 |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.5579        | 0.78  | 250  | 0.4547          | -0.0932        | -1.3103          | 0.8386             | 1.2171          | -222.2418      | -199.7473    | 0.5130          | 0.3441        |
+| 0.5578        | 0.78  | 250  | 0.4467          | -0.0981        | -1.3106          | 0.8410             | 1.2125          | -228.4777      | -209.0628    | 0.4528          | 0.2946        |
 
 
 ### Framework versions
 
-- PEFT 0.8.2
-- Transformers 4.37.2
-- Pytorch 2.2.0+cu121
-- Datasets 2.17.0
+- PEFT 0.9.0
+- Transformers 4.38.2
+- Pytorch 2.1.0+cu118
+- Datasets 2.18.0
 - Tokenizers 0.15.2

adapter_config.json

@@ -19,12 +19,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "fc1",
+    "k_proj",
     "fc2",
     "q_proj",
-    "k_proj",
-    "fc1",
     "v_proj"
   ],
   "task_type": "CAUSAL_LM",
+  "use_dora": false,
   "use_rslora": false
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9c945a28678bc285f469548170ea6c61f4ac181d365413b7e0be3345e0e70a9e
+oid sha256:0554899608cddc7554993c55ba320691f2dc5147a342d877731dbcc2946a7d8a
 size 167814424

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:16ab228666e6daad95fa26140b1b5ce43e3675dd453bc14833bdd9a7923cb0bd
-size 4728
+oid sha256:6af5bb41ca4173df8ec08e5e3443d56547ad580089b002e895a1309007ad0395
+size 4920