Abstract

Decision theory is the study of how rational agents should act in idealised situations. While it has important implications for the design and analysis of AI agents, there are still major unresolved problems. The two leading decision theories – Causal Decision Theory and Evidential Decision Theory – both seem to fail in some cases. Functional Decision Theory, an alternative proposed by researchers at MIRI , has some appealing properties but remains underspecified. In this talk I will explore the importance of decision theory to AI safety, explain the three theories mentioned above, and evaluate various objections which have been raised, and responses to them.

You can read more about the topic here: https://arxiv.org/abs/1710.05060

Talk slides: https://docs.google.com/presentation/d/1hoXUiEbGhzGcC2JkmcMPEpXgXL0Rj19hQdJn46zv1H8/edit#slide=id.p

https://valuealignment.ml/talks/2018-02-07-decision-theory-ai-safety.pdf