**Compressive memory.** Inspired by the plasticity in biological neurons (Munkhdalai and Yu, 2017; Miconi et al., 2018), compressive memory approaches cast parameterized functions as memory to store and retrieve information (Hinton and Plaut, 1987; Schmidhuber, 1992; Ba et al., 2016; Munkhdalai et al., 2019). Unlike the Transformer KV memory array (Vaswani et al., 2017; Wu et al., 2022), which grows with input sequence length, compressive memory systems maintain a constant number of memory parameters for computational efficiency. The parameters are modified with an update rule to store information, which is then retrieved via a memory reading mechanism (Graves et al., 2014; Sukhbaatar et al., 2015; Munkhdalai and Yu, 2017b).