BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Scaling Deep Learning - Misha Denil\, University of Oxford
DTSTART:20140321T100000Z
DTEND:20140321T110000Z
UID:TALK51319@talks.cam.ac.uk
CONTACT:Microsoft Research Cambridge Talks Admins
DESCRIPTION:Deep learning has seen an explosion of interest the the past f
 ew years due to some high profile success stories.  On a single machine GP
 Us are ideally suited to the computations required to train neural network
 s\, and have become the standard tool to accomplish this.  However\, scali
 ng neural networks beyond a single machine remains difficult\, as the cost
  of synchronizing weight updates quickly dominates the cost of computation
 .\n\nI will present two different approaches to dealing with communication
  overhead.  The first approach is based on the observation that filters in
  a neural network tend to be smooth\, and that the standard parameterizati
 on is wasteful in this case.  I will show how we can take advantage of thi
 s structure to re-parameterize a neural network to reduce the communicatio
 n overhead if the model is distributed over many machines.  The second app
 roach is a re-imagining of the standard neural network training algorithm 
 based on some recent advances in distributed parameter estimation in MRFs.
 \n
LOCATION:Small Lecture Theatre\, Microsoft Research Ltd\, 21 Station Road\
 , Cambridge\, CB1 2FB
END:VEVENT
END:VCALENDAR