Repository logo

Multinomial Thompson sampling for rating scales and prior considerations for calibrating uncertainty

Published version

Repository DOI

Change log


jats:titleAbstract</jats:title>jats:pBandit algorithms such as Thompson sampling (TS) have been put forth for decades as useful tools for conducting adaptively-randomised experiments. By skewing the allocation toward superior arms, they can substantially improve particular outcomes of interest for both participants and investigators. For example, they may use participants’ ratings for continuously optimising their experience with a program. However, most of the bandit and TS variants are based on either binary or continuous outcome models, leading to suboptimal performances in rating scale data. Guided by behavioural experiments we conducted online, we address this problem by introducing jats:italicMultinomial-TS</jats:italic> for rating scales. After assessing its improved empirical performance in unique optimal arm scenarios, we explore potential considerations (including prior’s role) for calibrating uncertainty and balancing arm allocation in scenarios with no unique optimal arms.</jats:p>


Acknowledgements: The author sincerely thanks Brunero Liseo for the constructive feedback on the manuscript, especially on the considerations on the Dirichlet prior parameters. The author also thanks Joseph Jay Williams and the IAI Lab for the motivating example and the MTurk I deployments data. Financial support by the NIHR Cambridge Biomedical Research Centre (BRC-1215-20014) is acknowledged.

Funder: Università degli Studi di Roma La Sapienza


49 Mathematical Sciences, 4905 Statistics, Mental health

Journal Title

Statistical Methods &amp; Applications

Conference Name

Journal ISSN


Volume Title



Springer Science and Business Media LLC