Expert Elicitation of Multinomial Probabilities for Decision-Analytic Modeling: An Application to Rates of Disease Progression in Undiagnosed and Untreated Melanoma.

Abstract

Background: Expert elicitation is required to inform decision making where relevant ‘better quality’ data either do not exist or cannot be collected. An example of this is to inform decisions as to whether to screen for melanoma. A key input is the counterfactual, in this case the natural history of melanoma in patients who are undiagnosed and hence untreated.

Objectives: To elicit expert opinion on the probability of disease progression in patients with melanoma that is undetected and hence untreated.

Methods: Bespoke webinar-based expert elicitation protocol administered to 14 participants in the UK, Australia and New Zealand, comprising 12 multinomial questions on the probability of progression from one disease stage to another in the absence of treatment. A modified Connor-Mosimann distribution was fitted to individual responses to each question. Individual responses were pooled using a Monte Carlo simulation approach. Participants were asked to provide feedback on the process.

Results: A pooled modified Connor-Mosimann distribution was successfully derived from participants’ responses. Feedback from participants was generally positive with 86% willing to take part in such an exercise again. However, only 57% of participants felt this was a valid approach to determine the risk of disease progression. Qualitative feedback reflected some understanding of the need to rely on expert elicitation in the absence of ‘hard’ data.

Conclusion: We successfully elicited and pooled the beliefs of experts in melanoma regarding the probability of disease progression in a format suitable for inclusion in a decision analytic model.