Repository logo

Research Matters 19


Recent Submissions

Now showing 1 - 8 of 8
  • ItemOpen AccessPublished version Peer-reviewed
    Research Matters 19: Winter 2015
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Green, Sylvia
    Research Matters is a free biannual publication which allows Cambridge University Press & Assessment to share its assessment research, in a range of fields, with the wider assessment community. 
  • ItemOpen AccessPublished version Peer-reviewed
    The moderation of coursework and controlled assessment: A summary
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Gill, Tim
    To ensure consistency and accuracy of marking, awarding bodies carry out moderation of GCSE and A level internally assessed work (e.g., coursework or controlled assessment). Training and instructions are provided by the awarding body to the internal assessors in each centre, including training in task-setting, marking and internal standardisation. Awarding bodies are required to modify centres' marks where necessary to bring judgements into line with the required standard. Samples are taken of (internally standardised) candidates' work. A moderator re-marks the sampled work, and if there is a difference between the centre's and moderator's marks that is larger than a certain amount then marks should be adjusted. Should it be necessary to adjust a centre's marks then the magnitude of the adjustments is determined by a regression analysis, based on the relationship between the marks given by the centre and those of the moderator in the sample. This article summarises the processes undertaken by the Oxford, Cambridge and RSA (OCR) exam board to moderate and, if necessary, adjust the marks of centre-marked coursework and controlled assessments. Some brief data analysis is also presented to give an idea of the extent of moderation and how much difference it makes to candidates' marks.
  • ItemOpen AccessPublished version Peer-reviewed
    Reflections on a framework for validation - Five years on
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Shaw, Stuart; Crisp, Vicki
    In essence, validation is simple. The basic questions which underlie any validation exercise are: what is being claimed about the test, and are the claims warranted (given all of the evidence). What could be more straightforward? Unfortunately, despite a century of theorising validity, it is still quite unclear exactly how much and what kind of evidence or analysis is required in order to establish a claim to validity. Despite Kane's attempts to simplify validation by developing a methodology to support validation practice, one which is grounded in argumentation (e.g., Kane, 1992), and the "simple, accessible direction for practitioners" (Goldstein & Behuniak, 2011, p.36) provided by the Standards (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education [AERA, APA, & NCME], 2014), good validation studies still prove surprisingly challenging to implement. In response, a framework for evidencing assessment validity in large-scale, high-stakes examinations and a set of methods for gathering validity evidence was developed in 2008/2009. The framework includes a number of validation questions to be answered by the collection of appropriate evidence and by related analyses. Both framework and methods were piloted and refined. Systematic implementation of the validation framework followed which employs two parallel validation strategies: 1. an experimental validation strategy which entails full post-hoc validation studies undertaken solely by research staff 2. an operational validation strategy which entails the gathering and synthesis of validation evidence currently generated routinely within operational processes. Five years on, a number of issues have emerged which prompted a review of the validation framework and several conceptual and textual changes to the language of the framework. These changes strengthen the theoretical structure underpinning the framework. This paper presents the revised framework, and reflects on the original scope of the framework and how this has changed. We also consider the suitability and meaningfulness of the language employed by the framework.
  • ItemOpen AccessPublished version Peer-reviewed
    Context led Science courses: A review
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Wilson, Frances; Evans, Steve; Old, Sarah
    Internationally, there is growing concern about secondary Science education. In many developed countries uptake of Science subjects has been falling, leading to fears that there will be a shortage of people with the scientific skills and knowledge needed in the twenty-first century. Science curricula are often considered to suffer from an overload of content, leading to the perception that Science subjects are among the most difficult. Furthermore, students have difficulties connecting the isolated facts which they are taught, and do not develop coherent mental schema. Content is often presented in an abstract manner that is remote from students' everyday experiences, so that many students do not understand why they should learn the materials which they are studying, and frequently fail to do so. As a result, students have difficulty applying scientific concepts in a context beyond the one in which they were taught that concept. In this article we examine one approach to Science education: context led Science courses, which have been developed as a result of these concerns.
  • ItemOpen AccessPublished version Peer-reviewed
    Assessing active citizenship: An international perspective
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Carroll, Prerna; Child, Simon; Darlington, Ellie
    This article analysed four education systems across five countries in terms of their approach to the assessment of 'active' citizenship. The four countries chosen were: England and Wales, Australia, Singapore and the USA (specifically Washington and Florida). 'Active' citizenship was defined as an amalgamation of knowledge (political literacy) and action (civic duty). As such, an assessment which tests both these constructs would be needed to provide a valid measure of active citizenship. For each education system, an overview of the placement of citizenship education is provided, alongside a critical analysis of the assessment of 'active' citizenship from the perspective of achieving test validity.
  • ItemOpen AccessPublished version Peer-reviewed
    An investigation into the numbers and characteristics of candidates with incomplete entries at AS/A level
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Vidal Rodeiro, Carmen
    AS and A levels are undergoing a comprehensive reform. In particular, from 2015, the AS level will be de-coupled from the A level and will become a standalone qualification, rather than being part of an A level. This move has raised concerns, as without a direct link to A levels, the new AS levels could reduce participation in subjects such as Mathematics or languages and may not be as beneficial. This research aimed to understand the numbers and types of students who start but do not complete their AS and A levels. Its outcomes will add to the debate surrounding the AS/A level reform and could help to anticipate changes in the uptake of the new AS levels. Furthermore, the subjects taken in the sixth form have implications for university recruitment and, therefore, a good understanding of how students refine their subject choice at the end of the first year of advanced study is crucial.
  • ItemOpen AccessPublished version Peer-reviewed
    A level History: Which factors motivate teachers' unit and topic choices?
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Child, Simon; Darlington, Ellie; Gill, Tim
    The flexibility inherent in A level History qualifications means that teachers have to negotiate competing factors that may influence topic, unit or qualification choices. The present article aimed to use questionnaire data derived from heads of History departments to analyse the motivations underpinning the unit and topic choices for an A level History course. A second aim was to analyse whether the Heads of Department from different school types had different influences underlying their choices. The two most common motivating factors underlying teachers’ choices of units and topics were found to be teacher expertise and perceived student engagement. Fisher’s Exact analyses revealed that these motivations were deemed significantly more important by state school teachers, compared to independent school teachers, in guiding their topic selections (both p < .05). There were also statistically significant differences between school types in terms of how their Heads of Department rated the importance of the curriculum support offered via different resources. These findings are discussed with reference to the recent qualifications reform in the UK, and the role of the teacher in determining topic choice and delivery of history.
  • ItemOpen AccessPublished version Peer-reviewed
    Text Mining: An introduction to theory and some applications
    (Research Division, Cambridge University Press & Assessment, 2015-12-01) Zanini, Nadir; Dhawan, Vikas
    Recent technological advances have led to the availability of new types of observations and measurements that were previously not available and that have fuelled the 'Big Data' trend (Dhawan & Zanini, 2014). Along with standard structured forms of data (containing mainly numbers), modern databases include new forms of unstructured data comprising words, images, sounds and videos which require new techniques to be exploited and interpreted. This article focusses on Text Mining (TM): a set of statistical and computer science techniques specifically developed to analyse text data. It aims to give a theoretical introduction to TM and to provide some examples of its applications.