An investigation into marker reliability and some qualitative aspects of on-screen marking

Abstract

There is a growing body of research literature that considers how the mode of assessment, either computer- or paper-based, might affect candidates' performances (Paek, 2005). Despite this, there is a fairly narrow literature that shifts the focus of attention to those making assessment judgements and which considers issues of assessor consistency when dealing with extended textual answers in different modes. This study involved 12 examiners marking 90 GCSE English Literature essays on paper and on screen and considered 6 questions: 1. Does mode affect marker reliability? 2. Construct validity - do examiners consider different features of the essays when marking in different modes? 3. Is mental workload greater for marking on screen? 4. Is spatial encoding influenced by mode? 5. Is navigation influenced by mode? 6. Is 'active reading' influenced by mode?