Abstract

In realistic choice tasks, especially sequential ones like mazes, actions are separated from their consequences by many steps of space and time. A central computational problem in decision making—which arises in various guises such as credit assignment and planning—is spanning these gaps to work out the long-term consequences of candidate actions. I review recent experimental and theoretical work aimed at understanding the mechanisms by which the brain solves this problem. First, I review a new study that monitors neural signatures of reward expectancy in rodents to monitor how the brain propagates information about individual experiences with outcomes to distal choicepoints. Second, I report ongoing theoretical work that aims to clarify how the brain can judiciously build and maintain cognitive maps so as to achieve effective decisions while minimizing computational costs. This offers a formal, resource-rational perspective on a range of issues such as habits and slips of action in the healthy brain, but also may explain dysfunctions such as compulsion, rumination, and avoidance.