What does information seeking tell us about reinforcement learning?Conventional theories of reinforcement learning explain how we choose actions to gain rewards, but we also often choose actions to help us predict rewards. This behavior is known as information seeking (or 'early resolution of uncertainty') in economics and a form of ‘observing behavior’ in psychology, and is found in both humans and animals. We recently showed that the preference to gather information about future rewards is signaled by many of the same neurons that signal preferences for appetitive rewards like food and water. This suggests that information seeking and conventional reward seeking share a common neural mechanism.
At the moment, we know very little about the nature of these neural computations. A major roadblock is theoretical: most prominent theories of reinforcement learning were originally designed to account for appetitive reward seeking and are unable to account for information seeking. How can we address this gap in our theories? I will summarize the state of the field, including my own work and that of others, and use this to propose ways that we can revise current theories of reinforcement learning to account for information seeking.