Repository logo
 

ChronoMID—Cross-modal neural networks for 3-D temporal medical imaging data

Published version
Peer-reviewed

Change log

Authors

Rakowski, Alexander G.  ORCID logo  https://orcid.org/0000-0002-7512-4169
Veličković, Petar 
Dall’Ara, Enrico 
Liò, Pietro 

Abstract

ChronoMID—neural networks for temporally-varying, hence Chrono, Medical Imaging Data—makes the novel application of cross-modal convolutional neural networks (X-CNNs) to the medical domain. In this paper, we present multiple approaches for incorporating temporal information into X-CNNs and compare their performance in a case study on the classification of abnormal bone remodelling in mice. Previous work developing medical models has predominantly focused on either spatial or temporal aspects, but rarely both. Our models seek to unify these complementary sources of information and derive insights in a bottom-up, data-driven approach. As with many medical datasets, the case study herein exhibits deep rather than wide data; we apply various techniques, including extensive regularisation, to account for this. After training on a balanced set of approximately 70000 images, two of the models—those using difference maps from known reference points—outperformed a state-of-the-art convolutional neural network baseline by over 30pp (> 99% vs. 68.26%) on an unseen, balanced validation set comprising around 20000 images. These models are expected to perform well with sparse data sets based on both previous findings with X-CNNs and the representations of time used, which permit arbitrarily large and irregular gaps between data points. Our results highlight the importance of identifying a suitable description of time for a problem domain, as unsuitable descriptors may not only fail to improve a model, they may in fact confound it.

Description

Keywords

Research Article, Computer and information sciences, Biology and life sciences, Research and analysis methods, Medicine and health sciences, Engineering and technology

Journal Title

PLOS ONE

Conference Name

Journal ISSN

1932-6203

Volume Title

15

Publisher

Public Library of Science
Sponsorship
Engineering and Physical Sciences Research Council (EP/K03877X/1)
National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC/R001073/1)