Statistical Inference and Learning for Stochastic and Partial Differential Equations

Abstract

Learning differential equation models from data is of significant interest to the scientific and engineering communities. Fundamental to many areas of science, differential equations are the mathematical description of change, derived from physical laws and modelling assumptions. Practically, however, mechanistic descriptions alone may prove insufficient to accurately model observed data; the model misspecification can be due to over-simplification, inaccurately specified parameters, or missing physics. The synthesis of mechanistic models with data-driven approaches promises to ameliorate these issues intrinsic to both modelling paradigms. In this thesis, we study statistical learning methods and their application to estimation problems for stochastic and partial differential equations. Through the use of recent statistical and machine learning approaches, we develop general methodologies for the estimation of stochastic and partial differential equation solutions and parameters from data. In our first contribution, we consider the nonparameteric estimation of stochastic differential equations (SDEs) for modelling time-series data, for the application of anomaly detection. Using Gaussian process based estimation of SDE coefficients, with marginal-likelihood optimisation for hyperparameter tuning, we derive a likelihood-ratio test for detecting anomalies. Our proposed algorithm, SDE-GPR, was shown to outperform baseline anomaly detection methods in discriminative power. The following contributions focus on the parametric estimation of partial differential equations from data, and the uncertainty quantification of the solution. The statistical finite element method (statFEM) underpins our methods, providing a stochastic representation of the mechanistic model, and generating a prior distribution over the PDE solution. Taking a Bayesian approach in both cases, we aim to quantify the uncertainty on the solution via the posterior distribution, whilst estimating unknown physical model parameters. Our second contribution considers the case of a known observation model. We directly target the posterior distribution with Langevin dynamics for sampling, with parameter estimation via maximum marginal likelihood methods. Convergence rates are derived and validated in the linear case, and both preconditioning and warm-start techniques are explored and shown to improve the convergence of the algorithm. In our final contribution, we extend this to the case of an unknown observation model, where data-driven machine learning methods --- specifically deep generative models --- are used to learn representations of unstructured data in a low-dimensional dynamical latent space. We develop the variational Bayesian methodology, Φ-DVAE, for joint estimation of the posterior and model parameters. The method was demonstrated on different linear, nonlinear and chaotic differential equation systems, for which we have enabled joint parameter and state inference with uncertainty quantification when observed data is unstructured.