University of Cambridge > Talks.cam > Machine Learning @ CUED > Compositional mathematics and automatic gradient descent

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Compositional mathematics and automatic gradient descent

Download to your calendar using vCal

If you have a question about this talk, please contact Dr R.E. Turner .

At its heart, deep learning involves composing operators and iteratively perturbing their weights. But we lack the mathematical tools needed to understand how compound operators behave under perturbation. As a consequence, our state-of-the-art training algorithms can be brittle and require manual tuning to work well. In this talk, we propose a new suite of mathematical tools for dealing with compound operators. This includes a new chain rule for understanding how “smoothness” or “linearisation error” behaves under composition, and also perturbation bounds for compound operators. We assemble these tools and apply the majorise-minimise principle to derive “automatic gradient descent”. AGD is a hyperparameter-free training algorithm for deep neural networks that has been validated at ImageNet scale and has the potential to further automate machine learning workflows.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Riley Auditorium, Clare College (Gillespie Centre, Memorial Court Site, Queen's Rd) Zoom webinar - link to follow Faculty of Classics

Other talks

Machine Learning for Spectral Geometry and Vice Versa Title to be confirmed The search for a new population of strong gravitational lenses with JWST CPH IRC and PHI Grand Rounds - Equine The Inaccessible Game
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity