Must examiners meet in order to standardise their marking? An experiment with new and experienced examiners of GCE AS Psychology

Abstract

When high-stakes examinations are marked by a panel of examiners, the examiners must be standardised so that candidates are not advantaged or disadvantaged according to which examiner marks their work. It is common practice for Awarding Bodies' standardisation processes to include a "Standardisation" or "Co-ordination" meeting, where all examiners meet to be briefed by the Principal Examiner and to discuss the application of the mark scheme in relation to specific examples of candidates' work. Research into the effectiveness of standardisation meetings has cast doubt on their usefulness, however, at least for experienced examiners. In the present study we addressed the following research questions:

What is the effect on marking accuracy of including a face-to-face meeting as part of an examiner standardisation process? How does the effect on marking accuracy of a face-to-face meeting vary with the type of question being marked (short-answer or essay) and the level of experience of the examiners? To what extent do examiners carry forward standardisation on one set of questions to a different but very similar set of questions?