Abstract

Hopfield networks, originally introduced in the 1980s, are recurrent neural networks that function as content-addressable memory systems. While classical Hopfield networks have limited capacity, modern variants leverage continuous states and attention-like energy functions to achieve exponential storage capacity. The influential work Hopfield Networks is All You Need bridges these advancements to transformer architectures, highlighting their significance in deep learning. In the first part of this talk, we will trace the evolution of Hopfield networks, examining their mathematical foundations and key applications in optimization. We will explore how these networks have transformed from their original binary state models to powerful continuous-state systems with deep learning applications. In the second part, we will step back to consider content-addressable memory in the brain, beginning with the hippocampal memory indexing theory. We will introduce a kernel-based formulation of key-value memory and discuss biologically plausible mechanisms for learning and organizing representations of queries, keys, and values. A key focus will be the recently proposed Vector-HaSH algorithm (Chandra et al., 2025, Nature), which offers a compelling model for efficient memory retrieval. Finally, we will review the main lines of evidence supporting key-value memory structures in the brain, drawing connections between neuroscience and modern machine learning.