ca-finetuned-phi-2

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.0636
+- Loss: 1.5643
 - Perplexity: 0.0000
 
 ## Model description
@@ -53,16 +53,16 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Perplexity |
 |:-------------:|:-----:|:----:|:---------------:|:----------:|
-| No log        | 1.0   | 1    | 2.4254          | 0.0000     |
-| No log        | 2.0   | 2    | 2.2526          | 0.0000     |
-| No log        | 3.0   | 3    | 2.2792          | 0.0000     |
-| No log        | 4.0   | 4    | 2.3126          | 0.0000     |
-| No log        | 5.0   | 5    | 2.2229          | 0.0000     |
-| No log        | 6.0   | 6    | 2.1234          | 0.0000     |
-| No log        | 7.0   | 7    | 2.1194          | 0.0000     |
-| No log        | 8.0   | 8    | 2.0873          | 0.0000     |
-| No log        | 9.0   | 9    | 2.0677          | 0.0000     |
-| No log        | 10.0  | 10   | 2.0636          | 0.0000     |
+| No log        | 0.97  | 7    | 2.2352          | 0.0000     |
+| No log        | 1.95  | 14   | 2.0164          | 0.0000     |
+| 2.4567        | 2.92  | 21   | 1.8511          | 0.0000     |
+| 2.4567        | 3.9   | 28   | 1.7391          | 0.0000     |
+| 1.8517        | 4.87  | 35   | 1.6667          | 0.0000     |
+| 1.8517        | 5.98  | 43   | 1.6059          | 0.0000     |
+| 1.5627        | 6.96  | 50   | 1.5816          | 0.0000     |
+| 1.5627        | 7.93  | 57   | 1.5672          | 0.0000     |
+| 1.41          | 8.9   | 64   | 1.5650          | 0.0000     |
+| 1.41          | 9.74  | 70   | 1.5643          | 0.0000     |
 
 
 ### Framework versions

adapter_config.json

@@ -19,8 +19,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "Wqkv",
     "fc2",
+    "Wqkv",
     "fc1"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ac06b08aa3c58f5b3ef3d293ab90a6e1217f6c4883299f5fc016cb81f56c7c5c
+oid sha256:4678bd7915cea1d44aae124d938dbe675f1042e6d7cacd741a1c8b92470f0143
 size 146825352

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:77e4c1d130952e564aabd0a0ecf001f41bac0cfe32d63a275e26af664ae090b5
+oid sha256:8bde404a3dd33164b6da9c608f71ba40e59153f8a3752fff3af18e84cfb38e7e
 size 4347