Given the absence of a common passing standard for students at UK medical schools, this paper compares independently -set standards for common “1 from 5” single-best answer (multiple choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences.

$Methods$: A repeated cross-sectional study, with participating schools sent a common set of graduation-level items (55 in 2013/14; 60 in 2014/1 5). Items were selected against a blueprint and underwent a quality review process. Each school employed their own standard setting process for the common items. The primary outcome was the passing standard for the common items for each medical school using the Angoff or Ebel methods.

$Results$: 22 (of 31 invited) medical schools participated in 2013/14 (71%) and 30 (97%) in 2014/15. Schools used a mean of 49 and 53 common items in 2013/14 and 2014/5 respectively; around one-third of the items in the examinations in which they were embedded. Data from 19 (61%) and 26 (84%) schools respectively met inclusion criteria for comparison of standards. There were statistically significant differences in the passing standard set by schools in both years (effect size (f$\mathrm{<mrow\; data-mjx-texclass="ORD">2</mrow>}$) 0.041 in 2013/14 and 0.218 in 2014/15, both p<0.001). The inter-quartile range of standards was 5.7 percentage points in 2013/14 and 6.5 percentage points in 2014/15. There was a positive correlation between the relative standards set by schools in the two years (Pearson’s r=0.57, n=18, p=0.014). Time allowed per item, method of standard setting and timing of exam in the curriculum did not have a statistically significant impact on standards.

$Conclusions$: Independently-set standards for common single-best answer items used in graduation-level examinations vary between UK medical schools. Further work to examine standard setting processes in more detail is needed to help to explain this variability and develop methods to help reduce it.