Toward Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach

Change log
Zhou, Y 
Johansen, AM 
Aston, JAD 

Model comparison for the purposes of selection, averaging, and validation is a problem found throughout statistics. Within the Bayesian paradigm, these problems all require the calculation of the posterior probabilities of models within a particular class. Substantial progress has been made in recent years, but difficulties remain in the implementation of existing schemes. This article presents adaptive sequential Monte Carlo (SMC) sampling strategies to characterize the posterior distribution of a collection of models, as well as the parameters of those models. Both a simple product estimator and a combination of SMC and a path sampling estimator are considered and existing theoretical results are extended to include the path sampling variant. A novel approach to the automatic specification of distributions within SMC algorithms is presented and shown to outperform the state of the art in this area. The performance of the proposed strategies is demonstrated via an extensive empirical study. Comparisons with state-of-the-art algorithms show that the proposed algorithms are always competitive, and often substantially superior to alternative techniques, at equal computational cost and considerably less application-specific implementation effort. Supplementary materials for this article are available online.

Adaptive Monte Carlo algorithms, Bayesian model comparison, Normalizing constants, Path sampling, Thermodynamic integration
Journal Title
Journal of Computational and Graphical Statistics
Conference Name
Journal ISSN
Volume Title
Informa UK Limited
Engineering and Physical Sciences Research Council (EP/K021672/2)
AJ was supported by EPSRC grant EP/I017984/1; JA by EPSRC grant EP/K021672/1 and the EPSRC/HEFCE CRiSM grant.