Is comparative judgement just a quick form of multiple marking?

Abstract

This article describes analysis of GCSE English essays that have both been scored using comparative judgement and marked multiple times. The different methods of scoring are compared in terms of the accuracy with which the resulting scores can predict achievement on a separate set of assessments. This results show that the predictive value of marking increases if multiple marking is used and (perhaps more interestingly) if statistical scaling is applied to the marks. More importantly, the evidence in this article suggests that any advantage of comparative judgement over traditional marking can be explained in terms of the number of judgements that are made for each essay and by the use of a complex statistical model to combine these. In other words, it is the quantity of data that is collected about each essay and how this data is analysed that is important. The physical act of placing two essays next to each other and deciding which is better does not appear to produce judgements that are in themselves any more valid than from getting the same individual to simply mark a set of essays.