Research Matters 14
Permanent URI for this collection
Browse
Recent Submissions
Item Open Access Published version Peer-reviewed Research Matters 14: June 2012(Research Division, Cambridge University Press & Assessment, 2012-06-01) Green, SylviaResearch Matters is a free biannual publication which allows Cambridge University Press & Assessment to share its assessment research, in a range of fields, with the wider assessment community.Item Open Access Published version Peer-reviewed The validity of teacher assessed Independent Research Reports contributing to Cambridge Pre-U Global Perspectives and Research(Research Division, Cambridge University Press & Assessment, 2012-06-01) Greatorex, Jackie; Shaw, StuartThis research considered the validity of tutor assessed, pre-university independent research reports. Evidence of construct relevance in tutors' interpretations of the levels awarded to the candidates' research process was investigated. This included designing, planning, managing and conducting their own research project using techniques and methods appropriate to the subject discipline. The research was conducted in the context of the Cambridge International Pre-U Global Perspectives and Independent Research qualification (the GPR), a pre-university qualification for 16-19 year olds which is designed to equip students with the skills required to make a success of their university studies. Tutors' justifications for the levels they gave candidates were considered. In the first of two studies (Study 1), tutor justifications were qualitatively analysed for specific tutor behaviours that might highlight tutors interpreting levels in a construct irrelevant way. In the second study (Study 2), external moderators (EMs) rated the justifications according to the extent to which they reflected the intended constructs. Study 1 showed little evidence of construct irrelevance and Study 2 provided strong evidence of construct relevance in tutors' interpretation of the levels they awarded candidates for the research process.Item Open Access Published version Peer-reviewed The effect of scripts’ profiles upon comparability judgements(Research Division, Cambridge University Press & Assessment, 2012-06-01) Rushton, NickyComparability studies often involve experts' judging students' scripts to decide which is better. Sometimes this involves experts judging more than one paper/component from a student. Students frequently achieve a higher level in one paper/component than they do on another. The scripts from these students are described as having an uneven profile. Uneven profile scripts have been identified as a cause of difficulty for those making judgements in comparability studies. This study investigated whether it was harder to compare uneven profiled scripts and whether uneven profiled scripts were judged more harshly. It found that the profile of the script only affected the difficulty of making comparisons in English Literature, where the effect varied by judge. Uneven profile scripts were slightly more likely to win their comparisons, but this depended on the judge. These findings suggest that the outcome of comparability studies could be affected by the existence of uneven profile scripts.Item Open Access Published version Peer-reviewed The Cambridge International Examinations bilingual research agenda(Research Division, Cambridge University Press & Assessment, 2012-06-01) Imam, Helen; Shaw, StuartThe context within which students prepare for Cambridge Assessment International Education (Cambridge International) assessments are often linguistically and educationally diverse. Whatever the country, the common denominator of Cambridge International schools is that students are being tutored and assessed through the medium of English. Some schools use bilingual instruction, delivering certain subjects through English as an additional language and other subjects through the first language, often trying to meet standards in both an international curriculum and a national curriculum. The opportunity to learn an additional language through a content subject has led to the practice of content and language integrated learning (CLIL) programmes. Other schools use monolingual instruction and deliver all subjects through English, either as a first or as an additional language. This article describes a Cambridge International research agenda designed to address four key questions: What is the impact of different teaching environments?; What impact does bilingual education have on the teaching and learning process?; What is the impact of bilingual education on learner outcomes?; What are the key assessment issues?Item Open Access Published version Peer-reviewed Piloting a method for comparing the demand of vocational qualifications with general qualifications(Research Division, Cambridge University Press & Assessment, 2012-06-01) Greatorex, Jackie; Shiell, HannahFrequently, researchers are tasked with comparing the demand of vocational and general qualifications, and methods of comparison often rely on human judgement. Therefore, the research aims to develop an instrument to compare vocational and general qualifications, pilot the instrument and explore how experts judge demand. Reading a range of OCR (Oxford Cambridge and Royal Society of Arts Examinations) level 2 specifications illustrated that they included knowledge, skills and understanding from five domains; the affective, cognitive, interpersonal, metacognitive, and psychomotor domains. Therefore, these domains were included in the instrument. Four cognate units were included in the study. Four experts participated, each with familiarity with at least one unit. Each expert read pairs of unit specifications and judged which was more demanding in each domain (affective, cognitive, interpersonal, metacognitive and psychomotor). Subsequently, they completed a questionnaire about their experience. The results are presented. It was found that the demands instrument was suitable for comparing the demand of cognate units from vocational and general qualifications.Item Open Access Published version Peer-reviewed Monitoring the difficulty of tiered GCSE components using threshold marks for grade C(Research Division, Cambridge University Press & Assessment, 2012-06-01) Dhawan, VikasThe study reported in this article aimed to explore simple ways of monitoring the relative difficulty of tiered components by considering the difference between the grade C boundaries on each tier. If the difficulty of the question papers is as intended, and the grade boundaries have been set correctly, the C boundary mark on the Foundation paper will be higher - as a proportion of the paper total - than the C boundary mark on the Higher paper. The difference between the C boundaries is easy to calculate and, when combined with other indicators, might prove useful for routine monitoring of the technical qualities of assessments. Thus in the present study we calculated these differences for two examination sessions, and we present our findings in this article.Item Open Access Published version Peer-reviewed An investigation on the impact of GCSE modularisation on A level uptake and performance(Research Division, Cambridge University Press & Assessment, 2012-06-01) Vidal Rodeiro, CarmenThe modularisation of GCSEs has caused considerable controversy since its introduction. Firstly, there are those who believe that modular assessment could lead to lack of coherence and fragmentation of learning as students have little time for reflection, skill development and knowledge consolidation; secondly, the increased assessment load means students could spend more time revising for the modular exams, rather than simply benefiting from the learning experience; and thirdly, there is the view that re-sitting modules may be lowering examination standards with 'teaching to the test' time heightened at the expense of deeper learning. Based on the above issues, teachers expressed concerns about modular students being less well equipped for the transition from GCSE to further study (e.g., A levels) than their linear counterparts. This study set out to investigate whether modular courses are good preparation for further study. The focus was on the impact of the GCSE assessment route on the uptake and performance in three A levels: English, mathematics and ICT.Item Open Access Published version Peer-reviewed Cambridge Assessment Statistics Reports: Recent highlights(Research Division, Cambridge University Press & Assessment, 2012-06-01) Emery, Joanne; Gill, Tim; Grayson, Rebecca; Vidal Rodeiro, CarmenThe Research Division publishes a number of Statistics Reports each year based on the latest national examinations data. These are statistical summaries of various aspects of the English examination system, covering topics such as subject provision and uptake, popular subject combinations, trends over time in the uptake of particular subjects and the examination attainment of different groups of candidates. The National Pupil Database (NPD) is the source of most of these reports. This is a very large longitudinal database, owned by the Department for Education, which tracks the examination attainment of all pupils within schools in England from their early years up to Key Stage 5 (A level or equivalent). Another database, the Pupil Level Annual School Census (PLASC), can be requested matched to the NPD. This contains background information on candidates such as deprivation indicators, language, ethnicity and special educational needs. Other sources of data used to produce the Statistics Reports include the Inter-Awarding Body Statistics produced by the Joint Council for Qualifications (JCQ). This article highlights some of the most recent Statistics Reports, published between 2010 and 2011.Item Open Access Published version Peer-reviewed An intra-board comparison at syllabus level based on outcomes of rank-ordering exercises at component level(Research Division, Cambridge University Press & Assessment, 2012-06-01) Yim, LouisThis study compared the use of scripts from 'pseudo-candidates' (a set of scripts from components of an assessment taken by different candidates) with scripts from real candidates (components all taken by the same candidate) to see if it affected the outcome of a comparability exercise using the rank-ordering method. The scripts from the real and pseudo-candidates were selected or created such that they had very similar profiles of marks (scores) across the components of the assessment.