Commit
•
16cd80e
1
Parent(s):
d7392c2
Update README.md
Browse files
README.md
CHANGED
|
@@ -28,7 +28,7 @@ effort.
|
|
| 28 |
|
| 29 |
Get access now at [LLM360 site](https://www.llm360.ai/)
|
| 30 |
|
| 31 |
-
## Model Description
|
| 32 |
|
| 33 |
- **Model type:** Language model with the same architecture as LLaMA-7B
|
| 34 |
- **Language(s) (NLP):** English
|
|
@@ -40,7 +40,7 @@ Get access now at [LLM360 site](https://www.llm360.ai/)
|
|
| 40 |
- [Fully processed Amber pretraining data](https://huggingface.co/datasets/LLM360/AmberDatasets)
|
| 41 |
|
| 42 |
|
| 43 |
-
# Loading Amber
|
| 44 |
|
| 45 |
To load a specific checkpoint, simply pass a revision with a value between `"ckpt_000"` and `"ckpt_358"`. If no revision is provided, it will load `"ckpt_359"`, which is the final checkpoint.
|
| 46 |
|
|
@@ -58,7 +58,7 @@ print(tokenizer.decode(outputs[0]))
|
|
| 58 |
|
| 59 |
```
|
| 60 |
|
| 61 |
-
# Amber Training Details
|
| 62 |
|
| 63 |
## DataMix
|
| 64 |
| Subset | Tokens (Billion) |
|
|
@@ -89,7 +89,7 @@ print(tokenizer.decode(outputs[0]))
|
|
| 89 |
| <img src="loss_curve.png" alt="loss curve" width="400"/> |
|
| 90 |
|
| 91 |
|
| 92 |
-
# Evaluation
|
| 93 |
|
| 94 |
Please refer to our [W&B project page](https://wandb.ai/llm360/CrystalCoder) for complete training logs and evaluation results.
|
| 95 |
|
|
@@ -101,7 +101,7 @@ Please refer to our [W&B project page](https://wandb.ai/llm360/CrystalCoder) for
|
|
| 101 |
|-----------------------------------------------------|-----------------------------------------------------------|
|
| 102 |
|<img src="amber-mmlu-curve.png" alt="mmlu" width="400"/> | <img src="amber-truthfulqa-curve.png" alt="truthfulqa" width="400"/> |
|
| 103 |
|
| 104 |
-
# Citation
|
| 105 |
|
| 106 |
**BibTeX:**
|
| 107 |
|
|
|
|
| 28 |
|
| 29 |
Get access now at [LLM360 site](https://www.llm360.ai/)
|
| 30 |
|
| 31 |
+
## 🟠 Model Description
|
| 32 |
|
| 33 |
- **Model type:** Language model with the same architecture as LLaMA-7B
|
| 34 |
- **Language(s) (NLP):** English
|
|
|
|
| 40 |
- [Fully processed Amber pretraining data](https://huggingface.co/datasets/LLM360/AmberDatasets)
|
| 41 |
|
| 42 |
|
| 43 |
+
# 🟠 Loading Amber
|
| 44 |
|
| 45 |
To load a specific checkpoint, simply pass a revision with a value between `"ckpt_000"` and `"ckpt_358"`. If no revision is provided, it will load `"ckpt_359"`, which is the final checkpoint.
|
| 46 |
|
|
|
|
| 58 |
|
| 59 |
```
|
| 60 |
|
| 61 |
+
# 🟠 Amber Training Details
|
| 62 |
|
| 63 |
## DataMix
|
| 64 |
| Subset | Tokens (Billion) |
|
|
|
|
| 89 |
| <img src="loss_curve.png" alt="loss curve" width="400"/> |
|
| 90 |
|
| 91 |
|
| 92 |
+
# 🟠 Evaluation
|
| 93 |
|
| 94 |
Please refer to our [W&B project page](https://wandb.ai/llm360/CrystalCoder) for complete training logs and evaluation results.
|
| 95 |
|
|
|
|
| 101 |
|-----------------------------------------------------|-----------------------------------------------------------|
|
| 102 |
|<img src="amber-mmlu-curve.png" alt="mmlu" width="400"/> | <img src="amber-truthfulqa-curve.png" alt="truthfulqa" width="400"/> |
|
| 103 |
|
| 104 |
+
# 🟠 Citation
|
| 105 |
|
| 106 |
**BibTeX:**
|
| 107 |
|