BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Mean-field dynamics and training of deep transformers - Christoph 
 Reisinger (University of Oxford)
DTSTART:20251110T155000Z
DTEND:20251110T163000Z
UID:TALK238441@talks.cam.ac.uk
DESCRIPTION:In this talk\, we will examine continuous limits of transforme
 r architectures\, which form the basis of common generative models. There 
 is rich literature on the limiting behaviour of neural networks\, includin
 g for large width of single layer neural networks (mean-field analysis) an
 d large depth of residual neural networks (neural ODE and SDE analysis). H
 ere\, we consider limits of transformers with attention and scaling for a 
 large number of layers\, tokens\, and attention heads. The analysis reveal
 s that for plausible training outputs\, a McKean--Vlasov limit with or wit
 hout diffusive common noise results. Joint work with William Gibson.
LOCATION:Seminar Room 1\, Newton Institute
END:VEVENT
END:VCALENDAR