BACKGROUND: Over recent years there has been a growth in cancer early diagnosis (ED) research, which requires valid measurement of routes to diagnosis and diagnostic intervals. The Aarhus Statement, published in 2012, provided methodological guidance to generate valid data on these key pre-diagnostic measures. However, there is still a wide variety of measuring instruments of varying quality in published research. In this paper we test comprehension of self-completion ED questionnaire items, based on Aarhus Statement guidance, and seek input from patients, GPs and ED researchers to refine these questions. METHODS: We used personal interviews and consensus approaches to generate draft ED questionnaire items, then a combination of focus groups and telephone interviews to test comprehension and obtain feedback. A framework analysis approach was used, to identify themes and potential refinements to the items. RESULTS: We found that many of the questionnaire items still prompted uncertainty in respondents, in both routes to diagnosis and diagnostic interval measurement. Uncertainty was greatest in the context of multiple or vague symptoms, and potentially ambiguous time-points (such as 'date of referral'). CONCLUSIONS: There are limits on the validity of self-completion questionnaire responses, and refinements to the wording of questions may not be able to completely overcome these limitations. It's important that ED researchers use the best identifiable measuring instruments, but accommodate inevitable uncertainty in the interpretation of their results. Every effort should be made to increase clarity of questions and responses, and use of two or more data sources should be considered.