File size: 261 Bytes
b9ef701 |
1 2 3 4 5 6 7 |
---
datasets:
- c4
language:
- en
---
CoreX models are Llama models in which the first X decoder layers are kept, and then the model is finetuned on 1 billion tokens from some dataset. Base model stems from Llama2-7b, medium from Llama2-13b, xl from Llama2-70b. |