Hugging Face's logo

mattshumer
/
Llama-3-8B-16K

Text Generation
Transformers
PyTorch
llama
text-generation-inference
Inference Endpoints
Model card Files Community
6
mattshumer commited on
Commit
5a67bfe
•
1 Parent(s): 77e34b7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -3,4 +3,6 @@ datasets:
3
  - Yukang/LongAlpaca-16k-length
4
  ---
5
 
6
- This is an extended (16K) context version of LLaMA 3. Trained for five hours on 8x A6000 GPUs, using the `Yukang/LongAlpaca-16k-length` dataset.
 
 
 
3
  - Yukang/LongAlpaca-16k-length
4
  ---
5
 
6
+ This is an extended (16K) context version of LLaMA 3. Trained for five hours on 8x A6000 GPUs, using the `Yukang/LongAlpaca-16k-length` dataset.
7
+
8
+ `rope_theta` was set to `1000000.0`. Trained with Axolotl.