BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Compositional mathematics and automatic gradient descent - Jeremy 
 Bernstein\, MIT
DTSTART:20230714T100000Z
DTEND:20230714T110000Z
UID:TALK203185@talks.cam.ac.uk
CONTACT:Dr R.E. Turner
DESCRIPTION:At its heart\, deep learning involves composing operators and 
 iteratively perturbing their weights. But we lack the mathematical tools n
 eeded to understand how compound operators behave under perturbation. As a
  consequence\, our state-of-the-art training algorithms can be brittle and
  require manual tuning to work well. In this talk\, we propose a new suite
  of mathematical tools for dealing with compound operators. This includes 
 a new chain rule for understanding how "smoothness" or "linearisation erro
 r" behaves under composition\, and also perturbation bounds for compound o
 perators. We assemble these tools and apply the majorise-minimise principl
 e to derive "automatic gradient descent". AGD is a hyperparameter-free tra
 ining algorithm for deep neural networks that has been validated at ImageN
 et scale and has the potential to further automate machine learning workfl
 ows.
LOCATION:Cambridge University Engineering Department\, CBL Seminar room BE
 4-38.
END:VEVENT
END:VCALENDAR