jrc commited on
Commit
c54ee96
1 Parent(s): 172f473

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -18,7 +18,13 @@ This is a Llama3 8B based model trained using [torchtune](https://pytorch.org/to
18
 
19
  ### Training details
20
 
21
- The exact training script (`lora_finetune_distributed`) and config (`8B_lora.yaml`) are both included in this repository. Specifically, in order to add the dataset, I added the following lines to the config:
 
 
 
 
 
 
22
 
23
  ```
24
  dataset:
@@ -30,4 +36,9 @@ train_on_input: False
30
  split: train
31
  ```
32
 
 
 
 
 
 
33
  ### Evaluation results
 
18
 
19
  ### Training details
20
 
21
+ The exact training script (`lora_finetune_distributed`) and config (`8B_lora.yaml`) are both included in this repository.
22
+
23
+ **Training command**: `tune run --nproc_per_node 8 lora_finetune_distributed --config 8B_lora.yaml`
24
+
25
+ > Yes I used 8 GPUs :)
26
+
27
+ In order to add the dataset, I added the following lines to the config:
28
 
29
  ```
30
  dataset:
 
36
  split: train
37
  ```
38
 
39
+ **Loss curve**
40
+
41
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/615b01ae487af9ad44dad803/Te9DycG2UVGm_JLnrG9De.png)
42
+
43
+
44
  ### Evaluation results