University of Cambridge > Talks.cam > Isaac Newton Institute Seminar Series > Algorithmic stability for heavy-tailed SGD

  Browse A-Z

Log in

University Account External (via Google)

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Algorithmic stability for heavy-tailed SGD

Download to your calendar using vCal

  • UserLingjiong Zhu (Florida State University)
  • ClockWednesday 24 April 2024, 14:30-15:15
  • HouseExternal.

If you have a question about this talk, please contact nobody.

TMLW02 - SGD: stability, momentum acceleration and heavy tails

Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error. In this study, we establish novel links between the tail behavior and generalization properties of stochastic gradient descent (SGD), through the lens of algorithmic stability. We develop generalization bounds for a general class of objective functions, which includes non-convex functions as well. Our approach is based on developing Wasserstein stability bounds for heavy-tailed SGD , which we then convert to generalization bounds, indicating a non-monotonic relationship between the generalization error and heavy tails. We support our theory with synthetic and real neural network experiments.   

This talk is part of the Isaac Newton Institute Seminar Series series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Average on weekdays: 8974.47931034 kWh Hybrid: in-person in Cambridge & streaming online, via YouTube Theo Chalmers Lecture Theatre , School of Clinical Medicine

Other talks

The uplift and erosion history of the Gamburtsev Subglacial Mountains and their role in the formation of the East Antarctic Ice Sheet Title and abstract tba Title to be confirmed Online Causal Inference Seminar Afternoon Break Zooming in on the assembly of galaxies in the early Universe
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity