Scaling prediction errors to reward variability benefits error-driven learning in humans
Journal of Neurophysiology
American Physiological Society
MetadataShow full item record
Diederen, K., & Schultz, W. (2015). Scaling prediction errors to reward variability benefits error-driven learning in humans. Journal of Neurophysiology, 114 1628-1640. https://doi.org/10.1152/jn.00483.2015
Effective error-driven learning requires individuals to adapt learning to environmental reward variability. The adaptive mechanism may involve decays in learning rate across subsequent trials, as shown previously, and rescaling of reward prediction errors. The current study investigated the influence of prediction error scaling and, in particular, the consequences on learning performance. Participants explicitly predicted reward magnitudes that were drawn from different probability distributions with specific standard deviations. By fitting the data using reinforcement learning models, we found scaling of prediction errors, in addition to the learning rate decay shown previously. Importantly, the prediction error scaling was closely related to learning performance, defined as accuracy predicting the mean of reward distributions, across individual participants. In addition, participants that scaled prediction errors relative to standard deviation also presented with more similar performance for different standard deviations, indicating that increases in standard deviation did not substantially decrease ‘adapters’ accuracy predicting the means of reward distributions. However, exaggerated scaling beyond the standard deviation resulted in impaired performance. Thus, efficient adaptation makes learning more robust to changing variability.
standard deviation, probability distribution, reinforcement learning, adaptation, risk
This work was supported by the Wellcome Trust and the Niels Stensen Foundation.
Wellcome Trust (093270/Z/10/Z)
Wellcome Trust (093875/Z/10/Z)
Wellcome Trust (095495/Z/11/Z)
External DOI: https://doi.org/10.1152/jn.00483.2015
This record's URL: https://www.repository.cam.ac.uk/handle/1810/248976
Attribution 2.0 UK: England & Wales
Licence URL: http://creativecommons.org/licenses/by/2.0/uk/