BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Geometry and Generalization: The case of Grokking - Matthieu Tehen
 an\, Department of Computer Sciences and Technology
DTSTART:20260428T110000Z
DTEND:20260428T120000Z
UID:TALK245788@talks.cam.ac.uk
CONTACT:Luning Sun
DESCRIPTION:Generalization in deep learning remains poorly understood\, pa
 rticularly in the feature-learning regime where representations are learne
 d jointly with predictors. Classical statistical learning theory provides 
 only limited explanations for the strong generalization observed in overpa
 rameterized models. The phenomenon of grokking\, where models achieve iden
 tical training performance yet generalize at different times\, offers a co
 ntrolled setting in which to study the mechanisms underlying generalizatio
 n. In this paper\, we propose that generalization can be explained through
  the geometry of learned representations. For tasks with known algebraic s
 tructure\, we show that the geometry of the ideal representation can be sp
 ecified analytically prior to training\, yielding what we call the geometr
 y of the task. In modular arithmetic\, this structure corresponds to the r
 egular representation of the cyclic group\, whose irreducible decompositio
 n defines a hierarchy of low-dimensional subspaces. We hypothesize that ge
 neralization occurs when a model’s internal representations align with t
 his universal geometry. We formalize this connection and derive bounds on 
 the generalization gap in terms of the geometric deviation between learned
  and ideal representations. Experiments support this view\, showing a crit
 ical threshold of geometric alignment beyond which generalization emerges 
 and demonstrating how regularization encourages convergence toward the uni
 versal geometry.
LOCATION:S3.05\, Simon Sainsbury Centre\, Cambridge Judge Business School
END:VEVENT
END:VCALENDAR