metadata
title: README
emoji: 🏢
colorFrom: pink
colorTo: blue
sdk: static
pinned: false
LLM360 is an initiative to open-source large language models (LLMs) that foster transparency, trust, and collaborative research.
LLM360 releases complete training details including all of the following artifacts associated with each model:
- Frequent Intermediate Model Checkpoints: During training, model parameters and optimizer states are collected regularly. These artifacts can offer valuable insights for studying LLM training dynamics and how it scales with data.
- Training Data with Full Data Sequence: The entire training dataset is fully disclosed and made publicly available. In addition, the full sequence of data examples used at each training step.
- Source Code: All the code used, including data processing, training, evaluation, and analysis.
- Logs and Metrics: All the training logs and evaluations done during training and using the final model are publicly disclosed.
LLM360 is a collaboration between Petuum, MBZUAI, and Cerebras.