Abstract

Decision-making in the face of uncertainty is a practical challenge found across various areas such as control and robotics, clinical trials, communication, and ecology. An extensively studied decision-making framework is that of stochastic contextual bandits (CBs) which uses side information, termed context, for sequential decision making. Prior research on CBs has mostly focussed on models where the contexts are well-defined. This, however, is not true in real-world applications where the contexts are either noisy or are indicative of predictive measurements. In this talk, we focus on noisy CBs where the learner observes only a noisy, corrupted, version of the true context through an unknown noise channel. We introduce a Thompson Sampling algorithm for Gaussian bandits with Gaussian context noise that can ‘approximate’ the action policy of an oracle which has access to the predictive distribution of the true context from the observed noisy context. Using information-theoretic tools, we study the Bayesian regret of the proposed algorithm.