Calculating the reliability of complex qualifications

Abstract

Most traditional methods of calculating reliability cannot be applied to complex qualifications that can be completed through multiple routes. For example, for some Maths A levels, candidates can choose when the take the various exam papers that are required, and, in addition, can choose the optional subjects in which they wish to complete exams. This article demonstrates a method by which reliability can be calculated in these instances by applying an optimal method of split halves to each individual assessment that may contribute to the qualification. All of these split halves can then be combined together to create "half qualifications" for each candidate regardless of the route they have taken. Once this have been achieved, traditional methods to calculate reliability can be applied. A full example from a Maths A level is provided.