Deep learning applied to hyperspectral endoscopy for online spectral classification.
Publication Date
2020-03-03Journal Title
Scientific reports
ISSN
2045-2322
Publisher
Nature Publishing Group
Volume
10
Issue
1
Pages
3947
Language
eng
Type
Article
This Version
AM
Physical Medium
Electronic
Metadata
Show full item recordCitation
Grigoroiu, A., Yoon, J., & Bohndiek, S. (2020). Deep learning applied to hyperspectral endoscopy for online spectral classification.. Scientific reports, 10 (1), 3947. https://doi.org/10.1038/s41598-020-60574-6
Abstract
Hyperspectral imaging (HSI) is being explored in endoscopy as a tool to extract biochemical information from tissue optical properties that may improve contrast for early cancer detection in the gastrointestinal tract. Motion artefacts during medical endoscopy have traditionally limited HSI application, however, recent developments in the field have led to real-time HSI deployments. Unfortunately, traditional HSI analysis methods remain unable to handle the volume of hyperspectral data in order to provide real-time feedback to the operator. Here, a convolutional neural network (CNN) is proposed to enable online classification of data obtained during HSI endoscopy. A five-layered CNN was trained and fine-tuned on a dataset of 300 hyperspectral endoscopy images acquired from a planar Macbeth ColorChecker chart and was able to distinguish between its 18 constituent colors with an average accuracy of 94.3% achieved at 8.8 fps. Performance was then tested on a set of images simulating an endoscopy environment, consisting of color charts warped inside a rigid tube
mimicking a lumen. The algorithm improved robust to such variations, with classification accuracies over 90% being obtained despite the variations, with an average drop in accuracy of 2.4% being registered at the points of longest working distance and most inclination. For further validation of the color-based classification system, ex vivo videos of a methylene blue dyed pig esophagus and images of different cancer
stages in the human esophagus were analyzed, showing spatially distinct color classifications. These results suggest that the CNN has potential to provide color-based classification during real-time HSI in endoscopy.
Sponsorship
The work was funded by the UK Engineering and Physical Sciences Research Council (EPSRC) (EP/N014588/1, EP/R003599/1),
CRUK (C47594/A16267, C14303/A17197, C47594/A21102) and the EU FP7 agreement FP7-PEOPLE-2013-CIG-630729.
AG was supported by the EPSRC grant for the University of Cambridge Centre for Doctoral Training in Sensor Technologies
and Applications (EP/L015889/1). We would like to thank Dr di Pietro, Dr Januszewicz and Prof. Fitzgerald of the MRC
Cancer Unit in Cambridge for their assistance in the original experiments that led to the generation of the pig esophagus and
tissue biopsy datasets.
Funder references
European Commission (630729)
Cancer Research UK (16267)
EPSRC (EP/N014588/1)
Cancer Research UK (C14303_do not transfer)
EPSRC (EP/R003599/1)
EPSRC (1783080)
EPSRC (EP/L015889/1)
Cancer Research UK (21102)
Embargo Lift Date
2023-02-13
Identifiers
External DOI: https://doi.org/10.1038/s41598-020-60574-6
This record's URL: https://www.repository.cam.ac.uk/handle/1810/302170
Rights
All rights reserved