This study examines variability across the age span in cognitive performance in a cross-sectional, population-based, adult lifespan cohort from the Cambridge Centre for Ageing and Neuroscience (Cam-CAN) study (n = 2680). A key question we highlight is whether using measures that are designed to detect age-related cognitive pathology may not be sensitive to, or reflective of, individual variability among younger adults. We present three issues that contribute to the debate for and against age-related increases in variability. Firstly, the need to formally define measures of central tendency and measures of variability. Secondly, in addition to the commonly addressed location-confounding (adjusting for covariates) there may exist changes in measures of variability due to confounder sub-groups. Finally, that increases in spread may be a result of floor or ceiling effects; where the measure is not sensitive enough at all ages. From the Cam-CAN study, a large population-based dataset, we demonstrate the existence of variability-confounding for the immediate episodic memory task; and show that increasing variance with age in our general cognitive measures is driven by a ceiling effect in younger age groups.