Research Matters 05


Recent Submissions

Now showing 1 - 9 of 9
  • ItemPublished versionOpen Access
    Research Matters 5: January 2008
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Green, Sylvia
    Research Matters is a free biannual publication which allows Cambridge University Press & Assessment to share its assessment research, in a range of fields, with the wider assessment community. 
  • ItemPublished versionOpen Access
    Using simulated data to model the effect of inter-marker correlation on classification consistency
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Gill, Tim; Bramley, Tom
    The marking of exam papers is never going to be 100% reliable unless all exams consist entirely of multiple-choice or other completely objective questions. Different opinions on the quality of the work or different interpretations of the mark schemes create the potential for candidates to receive a different mark depending on which examiner marks their paper. Of more concern for candidates is the potential for candidates to receive a different grade from a different examiner. The purpose of this study was to use simulated data to estimate the extent to which examinees might get a different grade for: i) different levels of correlation between markers and ii) for different grade bandwidths.
  • ItemPublished versionOpen Access
    The influence of performance data on awarders' estimates in Angoff awarding meetings
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Novakovic, Nadezda
    The Angoff method is one of the most widely used procedures for computing cut scores in both the vocational and general education settings. In the Angoff standard setting procedure, a panel of judges with subject expertise are asked to individually estimate, for each test item, the percentage of minimally competent or borderline candidates (MCCs) who would be able to answer that item correctly. The aim of this study was to investigate the relative effect of discussion and performance data on: (1) the awarders' expectations on how MCCs might perform on a test, (2) the magnitude of change in the awarders' estimates between sessions and (3) the awarders' rank-ordering of items in terms of their relative difficulty.
  • ItemPublished versionOpen Access
    Statistical reports: Patterns of GCSE and A-level uptake
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Emery, Joanne; Vidal Rodeiro, Carmen
    Two new statistical reports have been added to the 'Statistics Reports' series on the Cambridge Assessment website ( - Statistics Report Series No. 4: Uptake of GCSE subjects 2000 - 2006 - Statistics Report Series No. 5: Uptake of GCE A-Level subjects in England 2006 Data for these reports were extracted from the 16+/18+ databases. These databases are compiled for the Department for Children, Schools and Families from data supplied by all the awarding bodies in England. They contain background details and national examination data for all candidates who have their 16th, 17th and 18th birthdays in a particular school year. Candidates are allocated a unique number that remains the same throughout their Key Stage tests, allowing matching of examination data for longitudinal investigations. Records are present only if the candidate has sat an examination in a particular subject, not just attended classes. This brief article outlines some of the results from both reports.
  • ItemPublished versionOpen Access
    School-based assessment in international practice
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Johnson, Martin; Burdett, Newman
    The term 'school-based assessment' (SBA) can conjure up diverse and not necessarily synonymous meanings which often include forms of ongoing and continual classroom assessment of a formative nature. This article attempts to clarify how, and why, SBA has been successfully introduced in various contexts and the importance of the context in its success or otherwise. It reviews SBA research literature and lists some of the advantages of SBA, some of the reservations about using SBA, and some principles about when and why SBA should be used.
  • ItemPublished versionOpen Access
    Investigating the judgemental marking process: an overview of our recent research
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Suto, Irenka; Crisp, Vicki; Greatorex, Jackie
    This article gives an overview of a number of linked studies which explored the process of marking GCSE and A-level examination questions from a number of different angles. Key aims of these studies were to provide insights into how examiner training and marking accuracy could be improved, as well as reasoned justifications for how item types could be assigned to different groups of examiners in the future. The research studies combined several approaches, exploring both the information that people attend to when marking items and the sequences of mental operations involved. Examples include studies that used the think-aloud method to identify the cognitive marking strategies entailed in marking student responses, or to explore the broader socio-cognitive influences on the marking process. Other examples explored the relationship between cognitive marking strategy complexity and marking accuracy. This article brings together the findings from these various related studies to summarise the influences and processes that have been identified as important to the marking process from the research conducted so far.
  • ItemPublished versionOpen Access
    Independent examination boards and the start of a national system
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Watts, Andrew
    A presentation given as part of the programme for the 2007 Annual Alumni Weekend of the University of Cambridge, describing how public examinations were first introduced in England for secondary age students and charting the development of the system.
  • ItemPublished versionOpen Access
    An exploration of self-confidence and insight into marking accuracy among GCSE maths and physics markers
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Nadas, Rita; Suto, Irenka
    A considerable volume of literature in education and occupational research investigates issues in self-confidence and insight. However, GCSE markers' perceptions of their marking performance have not, to our knowledge, been examined. Exploring markers' perceptions is important for several reasons. First, if markers' estimates of their own performance prove to be accurate, then this information could be used by Awarding Bodies in standardisation procedures to identify and discuss examination questions that markers have difficulties with. If, however, markers' insight proves to be unreliable and unrelated to their actual marking accuracy, then their feedback on 'problem areas' could be misleading: for example, when conducting standardisation procedures, Principal Examiners might find themselves focussing on the 'wrong' questions. Secondly, investigating whether self-confidence and insight change or become more accurate with more marking practice or more feedback could inform marker training practices. In this article, we present research which explored GCSE markers' perceptions of their own marking accuracy. Markers' levels of self-confidence and insight, and possible changes in these measures over the course of the marking process, were investigated. The term 'self-confidence' here denotes markers' post-marking estimates of how accurately they thought they had marked a sample of questions; 'insight' refers to the relationship between markers' actual marking accuracy and estimated accuracy, indicating how precise their estimates were. Overall, we found very different patterns of self-confidence and insight for maths and physics markers.
  • ItemPublished versionOpen Access
    A review of literature regarding the validity of coursework and the rationale for its inclusion in the GCSE
    (Research Division, Cambridge University Press & Assessment, 2008-01-01) Crisp, Vicki
    Coursework was included in many GCSEs from their introduction in 1988 to increase the validity of assessment by providing wider evidence of student work and to enhance pupil learning by valuing skills such as critical thinking and independent learning (SEC, 1985). As the Secondary Examinations Council put it 'above all, the assessment of coursework can correspond much more closely to the scale of values in this wider world, where the individual is judged as much by his or her style of working and ability to cooperate with colleagues as by the eventual product' (SEC, 1985, p. 6). The validity and reliability of the assessment of GCSE coursework has come under much discussion since its introduction with the focus of concerns changing over time. At the inception of the GCSE, the main threats anticipated were possible unreliability of teacher marking, possible cheating and concern that girls were favoured (see QCA, 2006a). Now, concerns about consistency across similar subjects, fairness and authenticity (including the issues of internet plagiarism and excessive assistance from others), tasks becoming overly-structured (and hence reducing learning benefits) along with the overall burden on students across subjects, have led to a review of coursework by the Qualifications and Curriculum Authority (QCA). This article reviews relevant literature using the stages of assessment described by Crooks, Kane and Cohen (1996) to structure discussion of possible improvements to the validity of assessment as a result of including a coursework element within GCSE specifications and possible threats to validity associated with coursework.