LLaMA (Touvron et al., 2023) is a foundational, decoder-only large language model built upon the transformer architecture (Vaswani et al., 2017). Similar to the GPT series and other transformer-based LLMs, LLAMA consists of an embedding layer, multiple transformer blocks, and a language model head. LLAMA also incorporates improvements utilized in different models, such as pre-normalization (Zhang and Sennrich, 2019), SwiGLU activation (Shazeer, 2020), and rotary embeddings (Su et al., 2021). LLaMA is available in four different model sizes: 7B, 13B, 33B, and 65B.