Abstract

In 1973, Ferguson proposed to perform nonparametric estimation in a Bayesian framework by defining a prior distribution on an infinite-dimensional parameter space (the set of probability measures over a given domain). When applied to a finite set of observations, only a finite number out of the infinitely many degrees of freedom is used to explain the data, which accounts for the term “nonparametric”. The prior model is constructed as a stochastic process, with Dirichlet marginals and pathes in the set of probability measures over a separable metric space, that Ferguson called a “Dirichlet process”. His estimation model has a conjugate form with a closed-form solution for the posterior parameters, which mimics the conjugate posteriors of the Dirichlet marginals under a multinomial sampling model. Measures drawn at random from the Dirichlet process are a.s. discrete.

I will review Ferguson’s construction and his application of the model to sample observations. I also intend to briefly discuss the two major lines of research which developed from Ferguson’s paper: One that attempts to overcome the model’s discreteness in order to construct “universal” priors, and one that exploits discreteness to generalize the notion of finite mixtures and related models.

Article: http://www.ams.org/mathscinet-getitem?mr=350949

TS Ferguson, “A Bayesian analysis of some nonparametric problems” Ann. Statist. 1 (1973), 209—230.