BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Information-theoretic perspectives on learning algorithms - Dr Var
 un Jog\, University of Wisconsin-Madison
DTSTART:20180305T143000Z
DTEND:20180305T153000Z
UID:TALK102100@talks.cam.ac.uk
CONTACT:Prof. Ramji Venkataramanan
DESCRIPTION:In statistical learning theory\, generalization error is used 
 to quantify the degree to which a supervised machine learning algorithm ma
 y overfit to training data. We overview some recent work [Xu and Raginsky 
 (2017)] that bounds generalization error of empirical risk minimization ba
 sed on the mutual information between the algorithm input and the algorith
 m output. We leverage these results to derive generalization error bounds 
 for a broad class of iterative algorithms that are characterized by bounde
 d\, noisy updates with Markovian structure\, such as stochastic gradient L
 angevin dynamics (SGLD). We describe certain shortcomings of mutual inform
 ation-based bounds\, and propose alternate bounds that employ the Wasserst
 ein metric from optimal transport theory. We compare the Wasserstein metri
 c-based bounds with the mutual information-based bounds and show that for 
 a class of data generating distributions\, the former leads to stronger bo
 unds on the generalization error.\n\nThis is joint work with Adrian Tovar-
 Lopez\, Ankit Pensia\, and Po-Ling Loh
LOCATION:LT6\, Baker Building\, CUED
END:VEVENT
END:VCALENDAR