Repository logo
 

Order effects in high stakes undergraduate examinations: an analysis of 5 years of administrative data in one UK medical school.

Published version
Peer-reviewed

Change log

Authors

Barclay, Matt 
Evans, Robert 
Benson, John 

Abstract

OBJECTIVE: To investigate the association between student performance in undergraduate objective structured clinical examinations (OSCEs) and the examination schedule to which they were assigned to undertake these examinations. DESIGN: Analysis of routinely collected data. SETTING: One UK medical school. PARTICIPANTS: 2331 OSCEs of 3 different types (obstetrics OSCE, paediatrics OSCE and simulated clinical encounter examination OSCE) between 2009 and 2013. Students were not quarantined between examinations. OUTCOMES: (1) Pass rates by day examination started, (2) pass rates by day station undertaken and (3) mean scores by day examination started. RESULTS: We found no evidence that pass rates differed according to the day on which the examination was started by a candidate in any of the examinations considered (p>0.1 for all). There was evidence (p=0.013) that students were more likely to pass individual stations on the second day of the paediatrics OSCE (OR 1.27, 95% CI 1.05 to 1.54). In the cases of the simulated clinical encounter examination and the obstetrics and gynaecology OSCEs, there was no (p=0.42) or very weak evidence (p=0.099), respectively, of any such variation in the probability of passing individual stations according to the day they were attempted. There was no evidence that mean scores varied by day apart from the paediatric OSCE, where slightly higher scores were achieved on the second day of the examination. CONCLUSIONS: There is little evidence that different examination schedules have a consistent effect on pass rates or mean scores: students starting the examinations later were not consistently more or less likely to pass or score more highly than those starting earlier. The practice of quarantining students to prevent communication with (and subsequent unfair advantage for) subsequent examination cohorts is unlikely to be required.

Description

Keywords

Clinical Competence, Educational Measurement, MEDICAL EDUCATION & TRAINING, Adult, Clinical Competence, Confidence Intervals, Education, Medical, Undergraduate, Educational Measurement, Female, Humans, Male, Medical History Taking, Obstetrics, Pediatrics, Physical Examination, Reproducibility of Results, Schools, Medical, Students, Medical, United Kingdom

Journal Title

BMJ Open

Conference Name

Journal ISSN

2044-6055
2044-6055

Volume Title

6

Publisher

BMJ
Sponsorship
University of Cambridge School of Clinical Medicine