University of Cambridge > Talks.cam > Machine Learning @ CUED > Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

Log in

Google

Microsoft

Information on

Subscribing to talks details

Finding a talk details

Adding a talk details

Disseminating talks details

Help and Documentation details

Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

Download to your calendar using vCal

Diederik P. Kingma (University of Amsterdam)
Wednesday 14 October 2015, 11:00-12:00
Engineering Department, CBL Room BE-438.

If you have a question about this talk, please contact Dr Jes Frellsen .

We discuss a number of recent advances in Stochastic Gradient Variational Inference (SGVI).

Blending ideas from variational inference, deep learning and stochastic optimization, we derive an algorithm for efficient gradient-based inference and learning with intractable posteriors.

Applied to deep latent-variable models with neural networks as components, this results in the Variational Auto-Encoder (VAE), a principled Bayesian auto-encoder. We show that VAEs can be useful for semi-supervised learning and analogic reasoning.

Further improvements are realized through a new variational bound with auxiliary variables. Markov Chain Monte Carlo (MCMC) can be cast as variational inference with auxiliary variables; this interpretation allows principled optimization of MCMC parameters to greatly improve MCMC efficiency.

When applying SGVI to global parameters, we show how an order of magnitude of variance reduction can be achieved through local reparameterization while retaining parallelizability. Gaussian Dropout can be cast as a special case of such SGVI with a scale-free prior. This variational interpretation of dropout allows for simple optimization of dropout rates.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

📅 Download to calendar (vCal)

👤 Speaker: Diederik P. Kingma (University of Amsterdam) 🔗 Website
📅 Date & Time: Wednesday 14 October 2015, 11:00 - 12:00
📍 Venue: Engineering Department, CBL Room BE-438

Questions? Contact Dr Jes Frellsen

Abstract

We discuss a number of recent advances in Stochastic Gradient Variational Inference (SGVI).

Blending ideas from variational inference, deep learning and stochastic optimization, we derive an algorithm for efficient gradient-based inference and learning with intractable posteriors.

Applied to deep latent-variable models with neural networks as components, this results in the Variational Auto-Encoder (VAE), a principled Bayesian auto-encoder. We show that VAEs can be useful for semi-supervised learning and analogic reasoning.

Further improvements are realized through a new variational bound with auxiliary variables. Markov Chain Monte Carlo (MCMC) can be cast as variational inference with auxiliary variables; this interpretation allows principled optimization of MCMC parameters to greatly improve MCMC efficiency.

When applying SGVI to global parameters, we show how an order of magnitude of variance reduction can be achieved through local reparameterization while retaining parallelizability. Gaussian Dropout can be cast as a special case of such SGVI with a scale-free prior. This variational interpretation of dropout allows for simple optimization of dropout rates.

Series This talk is part of the Machine Learning @ CUED series.

Included in Lists

Note: Ex-directory lists are not shown.

Log in

🔐 Log In

Information on

ℹ️ Information

Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

This talk is included in these lists:

Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

Abstract

Included in Lists

Log in

🔐 Log In

Information on

ℹ️ Information

Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

This talk is included in these lists:

Other lists

Other talks

Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

Abstract

Included in Lists