Attention MechanismLLaMA 2 uses Grouped-Query Attention(GQA) [1] only on its largest 70B model, and its 7B and 13B uses full attention. We incorporate GQA in both Yi-6B and Yi-34B. GQA splits query-heads into G groups, sharing a single key and value head within each group of query [1]. This approach offers substantial reductions of training and inference costs, compared to the original Multi-Head Attention (MHA) [16; 57; 67]. We do not observe performance degradation after applying GQA to our 6B smaller model.