Structured machine learning for dynamical systems in healthcare

Abstract

Modelling the evolution of dynamical systems in healthcare is an important research area with many applications, ranging from determining the optimal duration of medical treatments to controlling the spread of an epidemic. There has been two dominate approaches for modelling dynamical systems. The first is the expert-driven approach, where human experts construct the equations based on specialised knowledge, scientific intuition, and personal experience. In this paradigm, the data takes a supportive role by informing hypothesis, validating models, or calibrating numerical constants. Due to the involvement of human experts, the expert-driven approach tends to be intensive in human resource and have long development cycles. It also falls short in problems involving many variables because it is a challenging task to manually identify or disentangle the effect of many variables. Recently, the advances of Machine Learning (ML) start to popularise the data-driven approach to dynamical system modelling, which employs automated algorithms to extract or approximate the governing equations from training data with minimum human inputs. Unlike the expert-driven approach, the data-driven approach requires large volumes of high-quality training data (data intensive). Furthermore, it usually generates opaque black-box models rather than compact and transparent equations, which limits its usability in high-stake applications. The aforementioned limitations are especially pronounced in healthcare applications, which often have high emergency, insufficient training data, many interacting variables, and strict requirements for model transparency. In this thesis, I will illustrate with examples that such scenarios are very common and they have high social and economical significance. To address these challenges, I develop a novel structured machine learning approach. The key insight is that there often exist imperfect expert equations that reveal some of the structural characteristics of the dynamical systems. The characteristics may include the temporal interaction structure between the (observed and latent) variables (e.g. A may influence B, but not C). They may also include the functional form of such interaction (e.g. quadratic form). Structured ML uses the available structural characteristics to inform algorithm design. The prior structural characteristics serve as a strong inductive bias for the ML algorithm, which enables sample-efficient learning and principled prediction. At the same time, the ML component makes it easy to include additional variables that were not included in the original expert equation, leading to a more comprehensive model. I applied the methods to two applications in managing COVID-19 pandemic. The first is to model the effect of non-pharmaceutical interventions on the COVID-19 fatality across different countries using early data. The second is to model the temporal effect of dexamethasone (a drug) on the immune activation of severally ill COVID-19 patients. In some applications however, there is no prior structural characteristic to leverage from. Hence, I proceed to develop automated algorithms for extracting structural characteristics from data, including the variable interaction structure and the functional form of the governing equation. These algorithms are aimed at supporting and accelerating the expert-driven approach. They are also suitable for applications which mandate compact and transparent equations. I applied the algorithms on a range of simulations and real applications, including modelling cancer relapse and discovering temporal comorbidity networks.