\(\bullet\)_Reducing data transfer_ mainly focuses on optimizing GPU memory access, thereby increasing the arithmetic intensity. As introduced in Section 4.2.2, KV cache can avoid redundant computation of previous tokens and PagedAttention allocates KV caches into continuous blocks to reduce memory fragmentation. Furthermore, Flash-Decoding [322] speeds up attention computation by loading the keys and values in parallel, especially effective for long text generation. As another alternative approach, multi-query and grouped-query attention can reduce the GPU memory bandwidth overhead by sharing KV parameters (loading fewer weights).