@inproceedings{
bhandarkar2023sequential,
title={Sequential Learning and Retrieval in a Sparse Distributed Memory: The K-winner Modern Hopfield Network},
author={Shaunak Bhandarkar and James Lloyd McClelland},
booktitle={Associative Memory {\&} Hopfield Networks in 2023},
year={2023},
url={https://openreview.net/forum?id=VOSrMFgWdL}
}
Many autoassociative memory models rely on a localist framework, using a neuron or slot for each memory. However, neuroscience research suggests that memories depend on sparse, distributed representations over neurons with sparse connectivity. Accordingly, we extend a canonical localist memory model---the modern Hopfield network (MHN)---to a distributed variant called the K-winner modern Hopfield network, equating the number of synaptic parameters (weights) in the localist and K-winner variants. We study both models' abilities to reconstruct once-presented patterns organized into long presentation sequences, updating the parameters of the best-matching memory neuron (or k best neurons) as each new pattern is presented.  We find that K-winner MHN's exhibit superior retention of older memories.
Sequential Learning and Retrieval in a Sparse Distributed Memory: The K-winner Modern Hopfield Network
Shaunak Bhandarkar James Lloyd McClelland