README / README.md
victormiller's picture
Update README.md
b4f09c6
|
raw
history blame
1.06 kB
metadata
title: README
emoji: 🏢
colorFrom: pink
colorTo: blue
sdk: static
pinned: false

LLM360 is an initiative to open-source large language models (LLMs) that foster transparency, trust, and collaborative research.

LLM360 releases complete training details including all of the following artifacts associated with each model:

  • Frequent Intermediate Model Checkpoints: During training, model parameters and optimizer states are collected regularly. These artifacts can offer valuable insights for studying LLM training dynamics and how it scales with data.
  • Training Data with Full Data Sequence: The entire training dataset is fully disclosed and made publicly available. In addition, the full sequence of data examples used at each training step.
  • Source Code: All the code used, including data processing, training, evaluation, and analysis.
  • Logs and Metrics: All the training logs and evaluations done during training and using the final model are publicly disclosed.​

LLM360 is a collaboration between Petuum, MBZUAI, and Cerebras.