baotnguyen/ncis

README.md

@@ -14,14 +14,13 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/phdatdt/Fine%20tuning%20mistral%207B/runs/2rd3zuba)
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/phdatdt/Fine%20tuning%20mistral%207B/runs/fgygb3c6)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/phdatdt/Fine%20tuning%20mistral%207B/runs/35ogk8js)
 # ncis
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.7540
-- Accuracy: 0.8641
+- Loss: 0.8124
+- Accuracy: 0.9223
 
 ## Model description
 
@@ -46,20 +45,22 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 8
+- num_epochs: 10
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Accuracy |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|
-| 0.7009        | 1.0   | 19   | 1.2681          | 0.7379   |
-| 0.0509        | 2.0   | 38   | 0.7540          | 0.8641   |
-| 0.0002        | 3.0   | 57   | 7.0541          | 0.4757   |
-| 0.0           | 4.0   | 76   | 1.1337          | 0.8252   |
-| 0.0           | 5.0   | 95   | 1.5852          | 0.7767   |
-| 0.0           | 6.0   | 114  | 2.2140          | 0.7379   |
-| 0.0           | 7.0   | 133  | 2.2768          | 0.7379   |
-| 0.0           | 8.0   | 152  | 2.2805          | 0.7379   |
+| 1.5187        | 1.0   | 19   | 0.8262          | 0.8155   |
+| 0.0217        | 2.0   | 38   | 0.8124          | 0.9223   |
+| 0.0304        | 3.0   | 57   | 1.5031          | 0.8544   |
+| 0.0           | 4.0   | 76   | 1.2126          | 0.8835   |
+| 0.0           | 5.0   | 95   | 1.1801          | 0.8835   |
+| 0.0           | 6.0   | 114  | 1.1762          | 0.8835   |
+| 0.0           | 7.0   | 133  | 1.1758          | 0.8835   |
+| 0.0           | 8.0   | 152  | 1.1757          | 0.8835   |
+| 0.0           | 9.0   | 171  | 1.1757          | 0.8835   |
+| 0.0           | 10.0  | 190  | 1.1757          | 0.8835   |
 
 
 ### Framework versions

adapter_config.json

@@ -23,9 +23,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj",
     "k_proj",
+    "v_proj",
+    "q_proj",
     "o_proj"
   ],
   "task_type": "SEQ_CLS",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:967188ee7af3ae290d5df30f2b0a0f5fd65613994f6481e32fec141bb1942551
+oid sha256:4b017a4411615b9287091cca85ea44fb6a6684b7187bf8a5a021313da204e875
 size 54593240

runs/Jul19_22-36-43_886d3e144ed7/events.out.tfevents.1721428604.886d3e144ed7.300.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc2cafb68a290d30f858013e2847db9759cb908ab969827af98fe8d457728eac
+size 11364

runs/Jul19_22-36-43_886d3e144ed7/events.out.tfevents.1721429746.886d3e144ed7.300.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32300cd0f8f447ca4729e6d59867617716fe9f8dd887b7c9f015efdba7dd1947
+size 411

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7b576aced4443d4ff4baa9ed31e2d0241403389eb9f2e5744016ae81a67b3020
+oid sha256:3398e53580df44d98ffd852b19b59a2ba70ccddb633b12420adcaa7b56d72129
 size 5112