Repository logo
 

Reinforcement Learning and Economic Decision-making for Nutrient Rewards in Rhesus Monkeys


Type

Thesis

Change log

Authors

Abstract

Value is a fundamental concept for economic decision-making and reinforcement learning. In these frameworks, the decision-makers choose by assigning values to objects and learn by updating object values with experience. However, it remains unclear how economic values derive from the intrinsic properties of objects and how subjective values of foods support adaptive food choices to maintain nutritional homeostasis. Here, I propose nutrients as biological sources of food values that link sensory food properties to nutrient decision-making. To investigate how economic values for food rewards derive from their nutrients and sensory qualities, I first demonstrated that monkeys consistently preferred fat and sugar to low-nutrient alternatives as if they assigned subjective values to the nutrient content. To identify the sensory basis of nutrient values, I developed engineering tools to measure food textures on oral-like biological surfaces. This approach revealed that the monkeys’ fat preferences can be explained by the subjective valuation of viscosity and oral sliding friction, suggesting an oral texture-sensing mechanism for nutrient valuation. In a formal ecological Geometric Framework for Nutrition (GFN), the monkeys’ food choices, guided by subjective nutrient values, shifted their nutrient balance away from dietary reference points, resembling human suboptimal eating in free-choice situations. Next, I examined how subjective nutrient values influence reinforcement learning in a dynamic food-choice environment. Monkeys learned faster from high-nutrient rewards and chose them frequently despite lower reward probabilities. This nutrient-differential learning can be modelled by a nutrient-sensitive reinforcement learning mechanism, which updates individual nutrient values from experience to derive integrated reward values that guide choices. Thus, nutrients constitute biological sources of economic values and reveal previously unrecognised aspects of reinforcement-learning mechanisms. Finally, to identify neuronal substrates of nutrient-based decision variables, I recorded extracellular single-neuron activities from the amygdala while monkeys evaluated nutrient-defined rewards cued by conditioned visual stimuli in a Pavlovian task. Amygdala neurons responded to visual conditioned stimuli by signalling the predicted rewards’ nutrient composition, sensory (textural) properties, and subjective values. In summary, I investigated primate nutrient-based reward mechanisms across behavioural, computational, and neuronal levels in a novel nutrient-choice paradigm. The findings show that nutrients and food textures shape the economic values of foods and guide adaptive food choices and learning in monkeys. These results point to behavioural and neural mechanisms behind human-like eating behaviour that could guide targeted neural interventions for obesity and eating disorders.

Description

Date

2021-09-30

Advisors

Schultz, Wolfram

Keywords

Reward, Nutrients, Reinforcement learning, Decision-making, Rhesus monkeys

Qualification

Doctor of Philosophy (PhD)

Awarding Institution

University of Cambridge
Sponsorship
Wellcome Trust (206207/Z/17/Z), Taiwan Ministry of Education (F-Y H)