Repository logo
 

Real-world testing of an artificial intelligence algorithm for the analysis of chest X-rays in primary care settings.

Published version
Peer-reviewed

Repository DOI


Change log

Authors

Miró Catalina, Queralt 
Vidal-Alaball, Josep 
Fuster-Casanovas, Aïna 
Escalé-Besa, Anna 
Ruiz Comellas, Anna 

Abstract

Interpreting chest X-rays is a complex task, and artificial intelligence algorithms for this purpose are currently being developed. It is important to perform external validations of these algorithms in order to implement them. This study therefore aims to externally validate an AI algorithm's diagnoses in real clinical practice, comparing them to a radiologist's diagnoses. The aim is also to identify diagnoses the algorithm may not have been trained for. A prospective observational study for the external validation of the AI algorithm in a region of Catalonia, comparing the AI algorithm's diagnosis with that of the reference radiologist, considered the gold standard. The external validation was performed with a sample of 278 images and reports, 51.8% of which showed no radiological abnormalities according to the radiologist's report. Analysing the validity of the AI algorithm, the average accuracy was 0.95 (95% CI 0.92; 0.98), the sensitivity was 0.48 (95% CI 0.30; 0.66) and the specificity was 0.98 (95% CI 0.97; 0.99). The conditions where the algorithm was most sensitive were external, upper abdominal and cardiac and/or valvular implants. On the other hand, the conditions where the algorithm was less sensitive were in the mediastinum, vessels and bone. The algorithm has been validated in the primary care setting and has proven to be useful when identifying images with or without conditions. However, in order to be a valuable tool to help and support experts, it requires additional real-world training to enhance its diagnostic capabilities for some of the conditions analysed. Our study emphasizes the need for continuous improvement to ensure the algorithm's effectiveness in primary care.

Description

Acknowledgements: The authors would like to thank all the users who agreed to participate in the study, and the radiology team at the Osona study centre for their help in recruiting patients. We would also like to thank the general practitioners for their participation in the interpretation of the radiologist's reports. Finally, we would also like to thank the professionals at Oxipit for their help in describing the more technical part of the algorithm. In addition, this study was carried out as part of the Industrial Doctorates programme of Catalonia and obtained a Bayès Grant.

Keywords

Algorithms, Artificial Intelligence, Primary Health Care, Radiography, X-Rays, Prospective Studies

Journal Title

Sci Rep

Conference Name

Journal ISSN

2045-2322
2045-2322

Volume Title

14

Publisher

Springer Science and Business Media LLC