University of Cambridge > Talks.cam > Machine Learning Reading Group @ CUED > Gradient-based Hyperparameter Optimisation

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Gradient-based Hyperparameter Optimisation

Download to your calendar using vCal

If you have a question about this talk, please contact Elre Oldewage .

We know the ultimate performance of our machine learning systems depends crucially on our training hyperparameters, motivating work to automate their selection. But traditional methods require many repeated runs and scale poorly to large numbers of hyperparameters (including variable schedules and per-parameter optimiser settings). New gradient-based methods aim to address these issues by updating hyperparameters during training itself, providing a more direct update signal than the black-box models used previously. In this talk, we explore the evolution of these methods and some recent developments, culminating in algorithms which can feasibly optimise millions of hyperparameters in parallel with network weights.

Optional Reading:

Our exposition will not assume any pre-reading. However, the following recent paper closely matches our notation and draws on much of the relevant literature, so would provide useful familiarisation for anybody who wishes:

Jonathan Lorraine, Paul Vicol, David Duvenaud, Optimizing Millions of Hyperparameters by Implicit Differentiation, AISTATS 2020

http://proceedings.mlr.press/v108/lorraine20a.html (avoid the out-of-date ArXiv version)

This talk is part of the Machine Learning Reading Group @ CUED series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

https://maths-cam-ac-uk.zoom.us/j/93998865836?pwd=VzVzN1VFQ0xjS3VDdlY0enBVckY5dz09 Zoom https://zoom.us/j/99492493911?pwd=TmRyMEpQazRyUlJVNDc5ZjRpa1M2dz09 Meeting ID: 994 9249 3911 Passcode: 603941 Engineering Department -LR4

Other talks

Title tbc Title TBC Beyond Hubble: Studying the Earliest Galaxies with the James Webb Space Telescope Psychiatry and Pain Medicine How Much Does Homology Really Know? Talk TBC
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity