Abstract

Suppose you publicly express your uncertainty about an unobserved quantity by quoting a distribution for it. A scoring rule is a special kind of loss function intended to measure the quality of your quoted distribution when an outcome is actually observed. In Bayesian decision theory, you seek to minimise your expected loss. A scoring rule is said to be proper if the expected loss under your quoted distribution is minimised by quoting that distribution. In other words, you cannot game the system! In addition to having a rich theoretical structure – for example, associated with every scoring rule is an entropy and a divergence function – scoring rules can be tailored to the problem at hand and consequently have a wide range of application. They are used in statistical inference, for evaluating and ranking weather and macroeconomic forecasters, for assessing the quality of predictive distributions, and in student examinations. I will discuss a class of scoring rules with the rather attractive property that the quoted distribution need only be known up to normalisation. This property is a consequence of requiring the scoring rule to be “local”, i.e. the score may depend on the quoted distribution at unrealised outcomes but only if those unrealised outcomes are “close” to the actual outcome. After completely specifying local scoring rules for both discrete and continuous outcome spaces, I will consider applications to missing data problems, the connection to the pseudolikelihood method, and the recent work of Hyvärinen et al. in machine learning. This is joint work with Philip Dawid and Steffen Lauritzen.