Comparing artificial intelligence and healthcare professional performance in surgical and interventional video analysis: a systematic review and meta-analysis.
Published version
Peer-reviewed
Repository URI
Repository DOI
Type
Change log
Abstract
This systematic review and meta-analysis examines the design of studies comparing the performance of artificial intelligence (AI) with that of healthcare professionals in the analysis of videos from surgical and interventional procedures, and quantitatively evaluates the performance of AI, unassisted healthcare professionals, and AI-assisted healthcare professionals. From the 37,956 studies identified, 146 were included, with 76 providing sufficient information for inclusion in our exploratory meta-analysis. AI had significantly greater sensitivity and comparable specificity compared to unassisted healthcare professionals at their respective peak performance levels, with a relative risk of 1.12 (95% CI 1.07-1.19, p < 0.001) and 1.04 (95% CI 0.98-1.10, p = 0.224), respectively. AI-assisted healthcare professionals had significantly greater sensitivity and specificity compared to unassisted healthcare professionals across all levels of expertise, with a relative risk of 1.18 (95% CI 1.12-1.25, p < 0.001) and 1.05 (95% CI 1.02-1.08, p < 0.001), respectively. There was no significant difference in sensitivity and specificity of AI-assisted expert healthcare professionals versus AI, with a relative risk of 0.99 (95% CI 0.95-1.04, p = 0.787) and 1.03 (95% CI 0.97-1.08, p = 0.395), respectively. Whilst most studies to date have evaluated AI head-to-head against unassisted healthcare professionals, fewer studies examined AI as an assistive tool, despite the real-world integration of AI more likely to involve assistance than autonomy.
Description
Acknowledgements: S.C.W. is supported by the Amethyst Healthcare Group. J.G.H. and D.Z.K. are supported by an NIHR Academic Clinical Fellowship. C.H.K. and D.Z.K. are supported by the Cleveland Clinic London PhD Fellowship. D.S. is supported by the Department of Science, Innovation and Technology, and the Royal Academy of Engineering under the Chair in Emerging Technologies programme. H.J.M. is supported by the NIHR Biomedical Research Centre at University College London. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The authors are grateful to Isla Kuhn (IK), medical librarian at the University of Cambridge, for reviewing and providing critique on the database search strategies. The authors are also grateful to Peter McCulloch, Professor of Surgical Science and Practice at the University of Oxford, for reviewing and providing critique on writing of the manuscript.
Publication status: Published
Funder: Amethyst Healthcare Group
Funder: NIHR Academic Clinical Fellowship
Funder: Cleveland Clinic London PhD Fellowship
Funder: Department of Science, Innovation and Technology
Funder: Royal Academy of Engineering under the Chair in Emerging Technologies programme
Funder: NIHR Biomedical Research Centre at University College London
Keywords
Journal Title
Conference Name
Journal ISSN
2398-6352
Volume Title
Publisher
Publisher DOI
Rights and licensing
