The application of Artificial Intelligence (AI) to the triage of suspicious skin lesions and the detection of skin cancer in primary care clinical settings

Abstract

The application of Artificial Intelligence (AI) to the triage of suspicious skin lesions and the detection of skin cancer in primary care clinical settings Background Skin cancers, including melanoma and the keratinocyte carcinomas (Basal Cell Carcinomas and Squamous Cell Carcinomas) are the most common cancers worldwide with rising incidence in most populations. Earlier diagnosis of skin cancer is associated with better outcomes for patients and improved survival. The first step in diagnosis of skin cancer is a patient, or their friend or relative, noticing a skin lesion that they are concerned about because of a change in appearance or new symptoms. In primary care, general practitioners (GPs) and other primary care practitioners then face the difficult task of triaging suspicious skin lesions to decide which lesions need onward referral on an urgent suspected skin cancer pathway for assessment in secondary care. There has been growing use of teledermatology services in recent years to triage urgent suspected skin cancer referrals from primary to secondary care, and discharge clearly benign skin lesions before they are seen in secondary care clinics. Artificial intelligence (AI) algorithms, primarily neural networks, have been applied to the diagnosis of skin cancer with promising diagnostic accuracy results; AI has been shown to be able to perform on a par with consultant dermatologists in the diagnosis of skin cancer. In the UK the current focus is on implementing AI in teledermatology settings, to help triage out clearly benign skin lesions referred from primary care and reduce the number of patients that need review in secondary care clinics. However, several questions remain about the efficacy, safety, and cost effectiveness of this approach. There are several alternative points in the skin cancer diagnostic pathway where AI could be implemented with various risks and benefits; implementation earlier in the pathway could help improve awareness and early presentation amongst the public and facilitate the earlier detection of skin cancer. Research is needed to understand the views of members of the public, patients, and healthcare professionals about the use of AI and its acceptability to them; and to understand how AI can best fit into the skin cancer diagnostic pathway. Aim In this thesis I aimed to explore the potential of AI technologies in primary care settings to triage suspicious skin lesions and detect skin cancer. I aimed to do this through establishing the current level of evidence for AI technologies in this setting, identifying potential barriers and facilitators, and exploring the views and preferences of patients, members of the public, healthcare professionals, and AI researchers. Methods First, I performed a systematic review to identify existing evidence on the use of AI algorithms to triage suspicious skin lesions and detect skin cancer, including the evidence for their use in primary care settings. This review included searching grey literature sources using scoping review methodology to identify commercially developed AI technologies that have no published evidence in the academic literature. Next, I undertook a qualitative interview study with patients, members of the public, GPs, primary care nurse practitioners, dermatologists, and AI researchers from both commercial and academic settings. This study was guided by the NASSS (non-adoption, abandonment, scale-up, spread, and sustainability) framework, and employed a thematic framework analysis approach with both inductive and deductive coding. Finally, I performed a discrete choice experiment (DCE) with patients, members of the public and GPs, evaluating preferences for AI attributes and the trade-off participants were willing to make between attributes. Using the data from this study I estimated the probability participants would be likely to select existing AI technologies over the gold-standard pathway of going to see their GP for assessment of a suspicious skin lesion. Results Whilst the review identified hundreds of studies reporting use of AI to triage suspicious skin lesions and detect skin cancer, there was very little evidence from primary care settings or settings with a low prevalence of skin cancer. There were also few implementation studies providing evidence on the performance of AI in real-life clinical settings. The review highlighted concerns about the representativeness of the datasets used to develop and test most AI algorithms, with data predominantly coming from tertiary dermatology centres in the US and Western Europe and from patients with melanin-poor skin, which suggests these AI algorithms may not generalise well, and their diagnostic accuracy may be lower when used in a general population. There was no clear consensus in the qualitative interviews on where AI should be positioned in the skin cancer diagnostic pathway, or for the best way to design an AI technology. Participants were primarily concerned about the risk of false negative results from the AI leading to missed diagnoses of skin cancer, but also discussed the risk that a technology with a low specificity could increase the workload of (already strained) healthcare services. Participants raised concerns which echoed issues highlighted in the systematic review, particularly regarding the quality and representativeness of the datasets used to train AI technologies and whether their diagnostic accuracy in real-life clinical practice would match that demonstrated in studies. Participants discussed the importance of AI technologies being easy to use to avoid excluding any sections of the population (e.g. older persons). Participants discussed the evaluation and regulation of AI technologies, including ongoing “sense checks” to monitor their performance over time. A low false negative rate was clearly the most important attribute for all participant groups in the DCE, echoing findings from the qualitative interview study. Developing and testing AI technologies on data from all skin tones and therefore ensuring safety with patients from all demographics, and a low false positive rate were also important to participants. Attribute preferences were similar between all participant groups, somewhat surprisingly participants from all groups preferred an AI technology that is used by an HCP in a GP surgery over an AI technology on a smartphone that patients could use at home. Discussion The aim of this thesis was to explore the potential of AI technologies in primary care clinical settings to triage suspicious skin lesions and facilitate the early detection of skin cancer. This thesis demonstrates that a gap remains to the implementation of AI in routine clinical practice. More evidence is needed evaluating AI technologies in clinical practice with attention paid to the quality and representativeness of the datasets used to develop and test them to ensure generalisability for all sections of the population and avoid inequity. There are several positions AI could fit into the skin cancer diagnostic pathway and several approaches they could take to facilitate the detection of skin cancer, however, the optimal route to implementation has not yet been established. Participants in the qualitative study expressed that using the AI technology with a “human-in-the-loop” was the preferred option for implementing these technologies initially. DCE participants preferred an AI technology that is used by a HCP in a GP surgery over an AI technology on a smartphone that patients could use at home. This likely follows-on from their desire to have a “human- in-the-loop” and could have important implications for implementation. A low false negative rate is clearly the most important attribute of AI technologies for developers and adopters to consider, to reduce the risk of missed skin cancer diagnoses. Accuracy of the AI technology in all demographics and a low false positive rate are also important attributes, to avoid inequity and reduce the risk of increasing the workload of healthcare services. Attribute preferences were similar between all participants groups; this is a positive finding for implementation because differing preferences and priorities could make implementation more difficult. The research is timely, given the interest in AI and the pressure the healthcare system is currently under, and highlights future research directions which could help ensure AI technologies designed to triage suspicious skin lesions and detect skin cancer achieve the potential they have demonstrated and do not worsen patient experiences and increase clinician workload.