Repository logo

How are standard-maintaining activities based on Comparative Judgement affected by mismarking in the script evidence?

Published version

Change log


Williamson, Joanna 


An important application of Comparative Judgement (CJ) methods is to assist in the maintenance of standards from one series to another in high stakes qualifications, by informing decisions about where to place grade boundaries or cut scores. This article explores the extent to which standard-maintaining activities based on Comparative Judgement would be robust to mismarking in the sample of scripts used for the comparison exercise. While extreme marking errors are unlikely, we know that mismarking can occur in live assessments, and quality of marking can vary. This research investigates how this could affect the outcomes of CJ-based methods, and therefore contributes to better understanding of the risks associated with using CJ-based methods for standard maintaining. The article focuses on the 'simplified pairs' method (Benton et al., 2020), an example of the 'universal method' discussed by Benton (this issue).



Comparative Judgement, Standards

Journal Title

Research Matters

Conference Name

Journal ISSN

Volume Title


Research Division, Cambridge University Press & Assessment

Publisher DOI

Publisher URL