In Infini-attention, instead of computing new memory entries for compressive memory, we reuse the query, key and value states (\(Q\), \(K\) and \(V\)) from the dot-product attention computation. The state sharing and reusing between the dot-product attention and compressive memory not only enables efficient plug-in-play long-context adaptation but also speeds up training and inference. Similar to the prior work (Munkhdalai et al., 2019), our goal is to store bindings of key and value states in the compressive memory and retrieve by using the query vectors.