Research Matters 13


Recent Submissions

Now showing 1 - 7 of 7
  • ItemPublished versionOpen Access
    Research Matters 13: January 2012
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Green, Sylvia
    Research Matters is a free biannual publication which allows Cambridge University Press & Assessment to share its assessment research, in a range of fields, with the wider assessment community. 
  • ItemPublished versionOpen Access
    The effect of manipulating features of examinees' scripts on their perceived quality
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Bramley, Tom
    Expert judgment of the quality of examinees' work plays an important part in standard setting, standard maintaining, and monitoring of comparability. In order to understand and validate methods that rely on expert judgment, it is necessary to know what features of examinees' work influence the experts' judgments. The controlled experiment reported here investigated the effect of changing four features of scripts from a GCSE Chemistry examination: i) the quality of written English; ii) the proportion of missing as opposed to incorrect responses; iii) the profile of marks in terms of fit to the Rasch model; and iv) the proportion of marks gained on the subset of questions testing 'good Chemistry'. Expert judges ranked scripts in terms of perceived quality. There were two versions of each script, an original version and a manipulated version (with the same total mark) where one of the four features had been altered. The largest effect was obtained by a combination of iii) and iv): increasing the proportion of marks gained on 'good Chemistry' items, and increasing the number of correct answers to difficult questions at the expense of wrong answers to easy questions. The implications of the findings for operational standard maintaining procedures are discussed.
  • ItemPublished versionOpen Access
    Making the most of our assessment data: Cambridge Assessment's Information Services Platform
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Raikes, Nick
    As new technologies penetrate every part of educational assessment, data is being collected as never before. Traditionally, there were two methods of producing statistical information within Cambridge Assessment. Routine statistical information came from reports built into bespoke examination processing systems. Non-routine analysis and reports were produced by small teams of statistical experts, typically working within research units and using statistical software packages on personal computers. With increasing demand for flexible, high-volume statistical reporting, a new solution was required; one which combined the resilience and scalability of a server-based infrastructure with the flexibility of having statistical experts in charge of creating the statistical content. The Information Services Platform (ISP) is Cambridge Assessment's solution for these requirements. It provides our statistical experts with access to operational assessment data and tools to automate and schedule analysis and report, and to publish the resulting content on an Intranet Portal for use by colleagues across the organisation. In this paper, I discuss further the thinking behind the ISP and give practical examples of use.
  • ItemPublished versionOpen Access
    Starting them young: research and project management opportunities for 16 to 19 year olds
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Suto, Irenka; Nadas, Rita
    Several educational routes have been developed which entail project work with a specific focus on independent learning and research. In this article we outline some of the options that exist at Level 3, primarily for 16 to 19 year olds (Years 12 and 13) in the UK and internationally. We then conduct a more detailed comparison of two routes: the Extended Project Qualification, and the International Baccalaureate Extended Essay. Many stakeholders may be unaware of the differences in the aims, structure, and scope of these routes. It is important for students and teachers to be conscious of the differences so that they can make informed decisions about what is most suitable for them. End-users such as higher education admissions tutors and employers also need to understand the differences in order to weigh up the experiences and achievements of applicants fairly.
  • ItemPublished versionOpen Access
    International assessment through the medium of English: analysing the language skills required
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Shaw, Stuart
    English is the medium of instruction and assessment for Cambridge Assessment International Education programmes of learning and summative assessment. Cambridge International programmes are increasingly offered in a variety of educational and bilingual contexts, and for many international students English is an additional language - their second language, or perhaps their third language. The Cambridge International context raises a number of issues relating to language awareness (e.g., progression from basic interpersonal communication skills to cognitive academic language proficiency) and assessment (e.g., the level of English needed to access and succeed in international assessments). The focus of the study is the International General Certificate of Secondary Education (IGCSE). The study adopts a two-phase methodology and involves an analysis of language use in Geography, History and Biology in order to (a) build a 'profile' of the language skills required and evidenced by IGCSEs and (b) determine whether any identifiable linguistic patterns adhere to different content, non-language IGCSEs. The level of English language proficiency required for accessing and succeeding in IGCSE is tentatively mapped to the Common European Framework of Reference for Languages (CEFR).
  • ItemPublished versionOpen Access
    An investigation into the number of special consideration enhancements and their impact on examination grades
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Vidal Rodeiro, Carmen
    Special consideration is a reasonable adjustment for students who were fully prepared for an examination and covered the course, but whose performance was affected by circumstances beyond their control (e.g., recent bereavement, serious accident, illness). This research investigated the number of special consideration requests (over time, by qualification and by school type) and the effect of the enhancements on the overall grades in a range of GCSE and A level subjects. The research shows that the number of special consideration applications rose in the period of study, with more requests at A level than at GCSE. Furthermore, candidates in independent schools were more likely to submit a request for special consideration than candidates in state schools. The special consideration enhancements considered in this work were minor adjustments to candidates' marks, with the most common tariff applied being 2% of the unit/component mark (normally due to minor illnesses at the time of the examination). As a result, the percentages of students improving their overall grades in GCSE and A level subjects were very small (lower than 1% in all subjects considered).
  • ItemPublished versionOpen Access
    An investigation into the impact of screen design on computer-based assessments
    (Research Division, Cambridge University Press & Assessment, 2012-01-01) Haigh, Matt
    This article reports on part of a study that investigated the impact of item format on the difficulty of test items. Two parallel forms of a computer-based test were developed; each test consisted of 15 items based on the GCSE Science curriculum. Five items were identical in both forms of the test to act as a control. The remaining items were modified in the parallel forms to investigate the effect of manipulating ten aspects of item format, including, for example, drag and drop categorisation vs. tick-box categorisation; and static graphic vs. animated graphic. The outcomes indicated that there was little effect on quantitative measures of item difficulty when the item format was changed.