Contrastive representation learning for bioimage quantification

Abstract

Deep learning has enabled unprecedented progress towards automating the analysis and quantification of large-scale, high-resolution imaging data. However, the majority of current deep learning systems for bioimage analysis is trained with manual annotations, leading to limitations in their generalization capabilities. To further automate exploration and quantification in imaging volumes, it is essential to identify inductive biases that enable the extraction of useful information, especially in settings where annotations are sparse or entirely absent. A promising approach to tackle this problem is self-supervised representation learning, which utilizes supervision from known priors or the data itself to tune the parameters of deep neural networks. In particular, self-supervised contrastive learning has shown remarkable success in domains such as vision and language. In this thesis, I extend and adapt contrastive learning techniques to address problems in bioimage analysis and beyond. I start by discussing representation learning, biological priors, and how to integrate them through contrastive learning, including data augmentations and similarity relationships. Further, I introduce the necessary technical background, including a more detailed discussion of contrastive joint embedding methods, the InfoNCE loss and its connection to mutual information estimation. The incorporation of priors through similarity relationships motivates the problem of leveraging multiple similar data samples for representation learning. As the first contribution, a principled generalization of the InfoNCE objective is proposed, called InfoNCEp, that can leverage multiple data pairs at once and is consistent with the noise contrastive estimation framework. I examine its theoretical properties, including its connection to mutual information maximization, and demonstrate in experiments its utility on the task of cell-type prediction in a dataset of spatial cell graphs. In the second contribution, the problem of supervised classification is addressed, which is a common final step in many data driven problems after an initial self-supervised pre-training procedure. Specifically, I combine soft targets with InfoNCE, resulting in a new supervised learning objective which is dubbed SoftInfoNCE. The new loss function can be interpreted as a temperature scaled energy cross-entropy, and I discuss its hard positive and hard negative mining properties. The conducted experiments demonstrate on two domains its superior performance over soft target and standard cross-entropy as well as InfoNCE. The thrid and final contribution is a novel self-supervised method for learning representations of cellular shape and texture from volume electron microscopy (EM) data. The method is applied to an EM volume of Platynereis dumerilii, and enables a visually consistent grouping of cells in the learned MorphoFeatures embedding space. Further, when MorphoFeatures are combined with features from spatial neighbours, tissues and organs can be retrieved. I specifically focus on shape embeddings, and describe the developed contrastive learning framework including the design of geometric augmentations and the networks used to process shapes. The complete pipeline can be seen as a stepping stone towards an automated exploration of large EM volumes.