commit

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [google/gemma-2-2b-it](https://huggingface.co/google/gemma-2-2b-it) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.8151
+- Loss: 0.5983
 
 ## Model description
 
@@ -46,29 +46,24 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.03
-- num_epochs: 2
+- training_steps: 600
 
 ### Training results
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 0.7509        | 0.1128 | 100  | 0.6985          |
-| 0.6275        | 0.2256 | 200  | 0.6195          |
-| 0.5875        | 0.3384 | 300  | 0.6070          |
-| 0.529         | 0.4512 | 400  | 0.6034          |
-| 0.5481        | 0.5640 | 500  | 0.6067          |
-| 0.495         | 0.6768 | 600  | 0.6103          |
-| 0.5166        | 0.7896 | 700  | 0.6325          |
-| 0.4658        | 0.9024 | 800  | 0.6337          |
-| 0.4582        | 1.0152 | 900  | 0.6718          |
-| 0.4279        | 1.1280 | 1000 | 0.6939          |
-| 0.3516        | 1.2408 | 1100 | 0.7089          |
-| 0.3698        | 1.3536 | 1200 | 0.7114          |
-| 0.3302        | 1.4664 | 1300 | 0.7501          |
-| 0.3319        | 1.5792 | 1400 | 0.7753          |
-| 0.3612        | 1.6920 | 1500 | 0.7918          |
-| 0.3763        | 1.8049 | 1600 | 0.8040          |
-| 0.3851        | 1.9177 | 1700 | 0.8151          |
+| 0.8045        | 0.0564 | 50   | 0.8067          |
+| 0.7271        | 0.1128 | 100  | 0.6777          |
+| 0.688         | 0.1692 | 150  | 0.6309          |
+| 0.6268        | 0.2256 | 200  | 0.6176          |
+| 0.572         | 0.2820 | 250  | 0.6118          |
+| 0.5864        | 0.3384 | 300  | 0.6065          |
+| 0.5528        | 0.3948 | 350  | 0.6030          |
+| 0.5396        | 0.4512 | 400  | 0.6015          |
+| 0.5726        | 0.5076 | 450  | 0.6005          |
+| 0.5655        | 0.5640 | 500  | 0.5997          |
+| 0.5712        | 0.6204 | 550  | 0.5988          |
+| 0.5213        | 0.6768 | 600  | 0.5983          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "gate_proj",
-    "q_proj",
     "k_proj",
-    "down_proj",
-    "v_proj",
+    "up_proj",
+    "q_proj",
     "o_proj",
-    "up_proj"
+    "gate_proj",
+    "down_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:029bac8f414dfedcccc81901b5e774d632980060b5ede475992ad8d14af5e003
+oid sha256:5ba1def0df12b2f74bf95cca48b4bc292b51bf3067d557fcdaf83e01a1946396
 size 83115256

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:87eed08dd27193b2ef5b85a8ad266f50a97d667444931f314821cb442d097e2c
-size 5432
+oid sha256:cb59d9a109654fee64ee77fdb560ba89ca39b157a9a318a55d76886e69d3de24
+size 5496