|
--- |
|
title: README |
|
emoji: 🏢 |
|
colorFrom: pink |
|
colorTo: blue |
|
sdk: static |
|
pinned: false |
|
--- |
|
|
|
|
|
**LLM360 is an initiative to open-source large language models (LLMs) that foster transparency, trust, and collaborative research.** |
|
|
|
LLM360 releases complete training details including all of the following artifacts associated with each model: |
|
|
|
* Frequent Intermediate Model Checkpoints: During training, model parameters and optimizer states are collected regularly. These artifacts can offer valuable insights for studying LLM training dynamics and how it scales with data. |
|
* Training Data with Full Data Sequence: The entire training dataset is fully disclosed and made publicly available. In addition, the full sequence of data examples used at each training step. |
|
* Source Code: All the code used, including data processing, training, evaluation, and analysis. |
|
* Logs and Metrics: All the training logs and evaluations done during training and using the final model are publicly disclosed. |
|
|
|
LLM360 is a collaboration between Petuum, MBZUAI, and Cerebras. |
