Learning Causal Representations for Generalization and Adaptation in Supervised, Imitation, and Reinforcement Learning

Abstract

This thesis studies the problem of learning causal representations, i.e., discovering low-dimensional high-level causal variables along with their causal relations from high-dimensional low-level observations, for generalization and adaptation in machine learning. First, we consider learning causal representations for generalization in supervised learning. Due to spurious correlations, predictive models often fail to generalize to environments whose distributions differ from the ones used at training time. To this end, we propose a framework, with theoretical guarantees under rather general assumptions over the underlying causal diagram, that first identifies direct causes of a given target from observations and then use those causes to build invariant predictors that are able to generalize to unseen testing environments. Second, we consider learning causal representations for generalization in imitation and reinforcement learning. A fundamental challenge therein is to learn policies, representations, or dynamics that do not build on spurious correlations and generalize beyond the specific environments that they were trained on. We investigate these generalization problems from a unified view. For this, we propose a framework to tackle them with theoretical guarantees on both identifiability and generalizability under mild assumptions on environmental changes. The key idea is that by taking advantage of structural relationships between environmental variables (i.e., observations, states, actions, and rewards), we first construct a data representation that ignores spurious features, and then build invariant predictors in terms of policy, representations, and dynamics. We theoretically show that the resulting policies, representations, and dynamics generalize well to unseen environments. Finally, we consider learning causal representations for adaptation in reinforcement learning. Apart from generalization, another fundamental challenge in reinforcement learning is how to quickly adapt a policy to new environments with only a few samples provided. By leveraging structural relationships over environmental variables, we construct a parsimonious graphical representation that encodes what and where a minimal and sufficient set of environment-specific factors and of environment-shared factors are respectively for policy adaptation. We show that such representations allow us to adapt the policy to the target environment in an efficient manner that requires only a few samples, without further policy optimization.