Abstract

Bayesian nonparametrics has the power to capture the infinite complexity in the real world, and could be used to explain how biological intelligence is capable of making extraordinary generalisations from a small number of examples. Gaussian processes are rich distributions over functions, which provide a Bayesian nonparametric approach to smoothing and interpolation. When the large nonparametric support of a Gaussian process is combined with the ability to automatically discover patterns in data and extrapolate, we are a step closer to developing truly intelligent agents, with applications in essentially any learning and prediction task.

In this talk, I derive simple closed form kernels that can be used with Gaussian processes, and other kernel machines, to discover patterns and enable extrapolation. These kernels are derived by modelling a spectral density—the Fourier transform of a kernel—with a Gaussian mixture. The proposed kernels support a broad class of stationary covariances, but Gaussian process inference remains simple and analytic. I demonstrate the proposed kernels by discovering patterns and performing long range extrapolation on synthetic examples, as well as atmospheric CO2 trends and airline passenger data. I also show that we can reconstruct standard covariances within the proposed framework.

This is joint work with Ryan P. Adams. A pre-print is available at http://arxiv.org/pdf/1302.4245v2, and through my website, http://mlg.eng.cam.ac.uk/andrew