BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:A Geometrical Perspective on Deep Neural Networks - Stanislav Fort
  (Stanford University\; formerly Google Research)
DTSTART:20200113T130000Z
DTEND:20200113T140000Z
UID:TALK136687@talks.cam.ac.uk
CONTACT:Ben Day
DESCRIPTION:Deep neural networks trained with gradient descent have been e
 xtremely successful at learning solutions to a broad suite of difficult pr
 oblems across a wide range of domains such as vision\, gameplay\, and natu
 ral language\, many of which had previously been considered to require int
 elligence. Despite their tremendous success\, however\, we still do not ha
 ve a detailed\, predictive understanding of how these systems work. In thi
 s talk\, I will focus on recent insights into the structure of neural netw
 ork loss landscapes and how they are navigated by gradient descent during 
 training. In particular\, I will discuss a phenomenological approach to mo
 delling their large-scale structure [1\,2]\, and its consequences for ense
 mbling\, calibration and Bayesian methods in general [3]. In addition\, I 
 will make a connection to empirical observations about loss gradients and 
 Hessians [4\,5]. I will conclude with an outlook on several interesting op
 en questions in understanding deep networks.\n\n# Fort\, Stanislav\, and A
 dam Scherlis. "The Goldilocks zone: Towards better understanding of neural
  network loss landscapes." Proceedings of the AAAI Conference on Artificia
 l Intelligence. Vol. 33. 2019. arXiv 1807.02581\n# Stanislav Fort\, and St
 anislaw Jastrzebski. "Large Scale Structure of Neural Network Loss Landsca
 pes." Advances in Neural Information Processing Systems 32 (NeurIPS 2019).
  arXiv 1906.04724\n# S Fort\, H Hu\, B Lakshminarayanan. "Deep Ensembles: 
 A Loss Landscape Perspective." arXiv 1912.02757\n# Stanislav Fort\, Paweł
  Krzysztof Nowak\, Stanislaw Jastrzebski\, Srini Narayanan. "Stiffness: A 
 New Perspective on Generalization in Neural Networks." arXiv 1901.09491\n#
  Stanislav Fort\, Surya Ganguli. "Emergent properties of the local geometr
 y of neural loss landscapes." arXiv 1910.05929  
LOCATION:LT2\, Computer Laboratory\, William Gates Building
END:VEVENT
END:VCALENDAR