Commit
•
7b8a52a
1
Parent(s):
32b8445
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,9 +1,7 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
-
|
| 5 |
-
license: apache-2.0
|
| 6 |
-
---
|
| 7 |
# LLM360 Research Suite: K2 Loss Spike 2
|
| 8 |
We encountered two major loss spikes while training K2.
|
| 9 |
* The [first loss spike](https://huggingface.co/LLM360/K2-Spike-1/) occured after X checkpoints and lasted over ~34 checkpoints. We restarted training at checkpoint X and training returned to normal.
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
|
|
|
|
|
|
|
| 5 |
# LLM360 Research Suite: K2 Loss Spike 2
|
| 6 |
We encountered two major loss spikes while training K2.
|
| 7 |
* The [first loss spike](https://huggingface.co/LLM360/K2-Spike-1/) occured after X checkpoints and lasted over ~34 checkpoints. We restarted training at checkpoint X and training returned to normal.
|