BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Efficient gradient coding for mitigating stragglers within distrib
 uted machine learning - Prof. Aditya Ramamoorthy\, Iowa State University
DTSTART:20251001T130000Z
DTEND:20251001T140000Z
UID:TALK235321@talks.cam.ac.uk
CONTACT:Prof. Ramji Venkataramanan
DESCRIPTION:*Please note that the room is different from the usual one*.\n
 \nLarge scale distributed learning is the workhorse of modern-day machine 
 learning algorithms. A typical scenario consists of minimizing a loss func
 tion (depending on the dataset) with respect to high-dimensional parameter
 . Workers typically compute gradients on their assigned dataset chunks and
  send them to the parameter server (PS)\, which aggregates them to compute
  either an exact or approximate version of the overall gradient of the rel
 evant loss function. However\, in large-scale clusters\, many workers are 
 prone to straggling (are slower than their promised speed or even failure-
 prone). A gradient coding solution introduces redundancy within the assign
 ment of chunks to the workers and uses coding theoretic ideas to allow the
  PS to recover the overall gradient (exactly or approximately)\, even in t
 he presence of stragglers. Unfortunately\, most existing gradient coding p
 rotocols are inefficient from a computation perspective as they coarsely c
 lassify workers as operational or failed\; the potentially valuable work p
 erformed by slow workers (partial stragglers) is ignored. \n\nIn this talk
  we will give an overview of some of our recent work in this area that add
 resses these limitations. Specifically\, we will present novel gradient co
 ding protocols that judiciously leverage the work performed by partial str
 agglers. Our protocols are simultaneously efficient from both a computatio
 n and communication perspective and numerically stable. For an important c
 lass of chunk assignments\, we present efficient algorithms for optimizing
  the relative ordering of chunks within the workers\; this ordering affect
 s the overall execution time. For exact gradient reconstruction\, our prot
 ocol is around 2x faster than the original class of protocols and for appr
 oximate gradient reconstruction\, the mean-squared-error of our reconstruc
 ted gradient is several orders of magnitude better.\n\n\n*Bio*: Aditya Ram
 amoorthy is the John Ryder Professor of Electrical and Computer Engineerin
 g and (by courtesy) of Mathematics at Iowa State University. He received h
 is B. Tech. degree in Electrical Engineering from the Indian Institute of 
 Technology\, Delhi and the M.S. and Ph.D. degrees from the University of C
 alifornia\, Los Angeles (UCLA). His research interests are in the areas of
  classical/quantum information theory and coding techniques with applicati
 ons to distributed computation\, content distribution networks and machine
  learning.\n\nDr. Ramamoorthy currently serves as an editor for the IEEE T
 ransactions on Information Theory (previous term from 2016—2019) and the
  IEEE Transactions on Communications from 2011—2015. He is the recipient
  of the Northrop Grumman professorship (2022-24)\, the 2020 Mid-Career Ach
 ievement in Research Award\, the 2019 Boast-Nilsson Educational Impact Awa
 rd and the 2012 Early Career Engineering Faculty Research Award from Iowa 
 State University\, the 2012 NSF CAREER award\, and the Harpole-Pentair pro
 fessorship in 2009-2010.\n
LOCATION:MR9\, CMS Pavilion B
END:VEVENT
END:VCALENDAR