Abstract

Abstract: While years of scientific research on model training and scaling assume that learning is a gradual and continuous process, breakthroughs on specific capabilities have drawn wide attention. Why are breakthroughs so exciting? Because humans don’t naturally think in continuous gradients, but in discrete conceptual categories. If artificial language models naturally learn discrete conceptual categories, perhaps model understanding is within our grasp. I will describe what we know of categorical learning in language models, and how discrete concepts are identifiable through empirical training dynamics and through random variation between training runs. These concepts involve syntax learning, weight mechanisms, and interpretable patterns—-all of which can predict model behavior. By leveraging categorical learning, we can ultimately understand a model’s natural conceptual structure and evaluate our understanding through testable predictions.

Bio: Naomi Saphra is a research fellow at the Kempner Institute at Harvard University and incoming faculty at Boston University in 2026. Naomi is interested in empirically understanding training in NLP and language models: how models learn to encode linguistic patterns or other structure and how we can encode useful inductive biases into the training process. Recently, she has begun collaborating with natural and social scientists to use interpretability to understand the world around us. She is particularly interested in fish. Previously, she earned a PhD from the University of Edinburgh on Training Dynamics of Neural Language Models; worked at NYU , Google, MosaicML, and Facebook; and attended Johns Hopkins and Carnegie Mellon University. Outside of research, she plays roller derby under the name Gaussian Retribution and performs standup comedy.