BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Algorithmic stability for heavy-tailed SGD - Lingjiong Zhu (Florid
 a State University)
DTSTART:20240424T133000Z
DTEND:20240424T141500Z
UID:TALK214186@talks.cam.ac.uk
DESCRIPTION:Recent studies have shown that heavy tails can emerge in stoch
 astic optimization and that the heaviness of the tails have links to the g
 eneralization error. In this study\, we establish novel links between the 
 tail behavior and generalization properties of stochastic gradient descent
  (SGD)\, through the lens of algorithmic stability. We develop generalizat
 ion bounds for a general class of objective functions\, which includes non
 -convex functions as well. Our approach is based on developing Wasserstein
  stability bounds for heavy-tailed SGD\, which we then convert to generali
 zation bounds\, indicating a non-monotonic relationship between the genera
 lization error and heavy tails. We support our theory with synthetic and r
 eal neural network experiments.&nbsp\;\n&nbsp\;
LOCATION:External
END:VEVENT
END:VCALENDAR