University of Cambridge > Talks.cam > Machine Learning @ CUED > Using gradient descent for optimization and learning

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Using gradient descent for optimization and learning

Download to your calendar using vCal

If you have a question about this talk, please contact Simon Lacoste-Julien .

In machine learning, to solve a particular task, we often define a cost function which we are trying to minimize over a set of data points, the training set. There has been extensive work on designing efficient optimization techniques to minimize such functions, amongst them BFGS and conjugate gradient descent. However, this only ensures a fast decrease in the training error, whereas we are ultimately interested in minimizing another error: the test error. Reaching a low test error for a particular problem is called learning. As Bottou has shown, good optimizers may prove to be very bad learning methods and vice-versa.

After reviewing the most popular optimization techniques, I shall make a brief summary of Bottou’s conclusions and then present a new gradient descent algorithm which aims at directly addressing this issue of optimizing the test error using only a training set.

Speaker’s bio:

Nicolas received a Master’s Degree in Applied Maths from the Ecole Centrale Paris and one in Maths, Vision and Learning from the ENS Cachan in 2003. From 2004 to 2008, he did a PhD in Montreal under the supervision of Yoshua Bengio, working on designing and optimizing neural networks. Since 2008, he is a postdoc researcher at Microsoft Research Cambridge, working with John Winn on using deep neural networks for vision.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Cancer Research UK Cambridge Institute, Lecture Theatre CUUEG Talks Engineering Department, CBL Room 438

Other talks

"That's History. That's Truth. I Seen It Myself": T'tc~tsa (Wailaki) and the Enslavement of California Native Americans. TBC CSAR lecture: In a post-Ozempic world, have we cured obesity? Dr Dinis Calado, Senior Group Leader, Immunity and Cancer Laboratory, the Francis Crick Institute Complete and Submit Project Choice Forms The Magic of Magnetism
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity