Transparent machine learning for scientific modelling beyond closed-form equations

Abstract

Scientific progress relies on models that allow us to describe, understand, and predict the behaviour of real-world phenomena. Mathematical equations form the backbone of many scientific models, offering a formal and compact representation of our knowledge. Traditionally, such equations were proposed or derived by human experts. Recently, however, machine learning (ML) methods have been used to discover equations directly from data. A prominent example is symbolic regression (SR), which searches for predictive models in the form of mathematical expressions. Although SR has been successful in uncovering many well-known equations from physics, chemistry, and biology, its performance becomes less certain when applied to complex real-world datasets that do not lend themselves to concise analytical descriptions. Yet for many such datasets (e.g., in medicine), fully transparent models are particularly valuable and often necessary. In this thesis, I propose new classes of ML models that retain key advantages of mathematical equations while not being constrained to compact purely symbolic expressions. As a result, they may be more flexible and applicable to real-world settings. First, I introduce a mathematical framework to characterise what makes some equations easy to analyse. Then I build on these insights to propose a new model class that extends SR by univariate shape functions from generalised additive models, thereby unifying the two approaches. The second half of this work focuses on dynamic settings (traditionally addressed by ordinary differential equations). First, I investigate what it means for a time series forecasting method to be fully transparent. The next chapter leverages those insights to propose a new approach to modelling dynamical systems called direct semantic modelling. In contrast to the traditional two-step modelling approach, where an ordinary differential equation is first found and then analysed, this framework directly outputs the description of the system behaviour, making it easier to understand, verify and edit. I introduce Semantic ODE, an instantiation of the framework for one-dimensional systems. Finally, I extend it to multidimensional dynamical systems that may depend on auxiliary static features.