University of Cambridge > Talks.cam > Machine Learning @ CUED > Fast yet Simple Natural-Gradient Variational Inference in Complex Models

Log in

Google

Microsoft

Information on

Subscribing to talks details

Finding a talk details

Adding a talk details

Disseminating talks details

Help and Documentation details

Fast yet Simple Natural-Gradient Variational Inference in Complex Models

Download to your calendar using vCal

Emtiyaz Khan, team leader (equivalent to Full Professor) at the RIKEN center for Advanced Intelligence Project (AIP) in Tokyo
Monday 16 July 2018, 11:00-12:00
Engineering Department, CBL Room BE-438..

If you have a question about this talk, please contact Dr R.E. Turner .

Approximate Bayesian inference is promising in improving generalization and reliability of deep learning, but is computationally challenging. Modern variational-inference (VI) methods circumvent the challenge by formulating Bayesian inference as an optimization problem and then solving it using gradient-based methods. In this talk, I will argue in favor of natural-gradient approaches which can improve convergence of VI by exploiting the information geometry of the solutions. I will discuss a fast yet simple natural-gradient method obtained by using a duality associated with exponential-family distributions. I will summarize some of our recent results on Bayesian deep learning, where natural-gradient methods lead to an approach which gives simpler updates than existing VI methods while performing comparably to them.

Joint work with Wu Lin (UBC), Didrik Nielsen (RIKEN), Voot Tangkaratt (RIKEN), Yarin Gal (UOxford), Akash Srivastva (UEdinburgh), Zuozhu Liu (SUTD).

Based on:

https://emtiyaz.github.io/papers/isita2018_preprint.pdf

https://arxiv.org/abs/1806.04854

https://arxiv.org/abs/1703.04265

This talk is part of the Machine Learning @ CUED series.