Learning-based cooperative perception and control for multi-robot systems

Abstract

Multi-robot systems (MRS) are key to addressing complex real-world problems through the coordination of multiple autonomous agents but are burdened by the curse of dimensionality, which compounds the difficulty of applying solutions to large-scale problem instances. The use of learning-based methods in multi-robot planning holds great promise as it enables us to offload the online computational burden of expensive centralised, yet optimal solvers, to an offline learning procedure. Recent work has shown the promise of Graph neural networks (GNNs) to learn explicit communication strategies that enable complex multi-agent coordination. In the context of MRS, individual robots are modelled as nodes, the communication links between them as edges, and the internal state of each robot as graph signals. Sending messages over the communication links allows each robot in the graph to obtain an improved approximation of the global state, compared to relying on local information. However, translating these policies from theoretical models to practical real-world MRS remains a critical challenge. Expert data is typically generated in simulations, yet policies trained in simulation often do not generalise to the real world. This lack of transferability is referred to as the reality gap, and transferring policies across this gap is a key challenge referred to as sim-to-real transfer. While sim-to-real transfer in the single-robot domain typically deals with physical robot-world interaction, the multi-robot domain also concerns robot-robot interactions. Communication is vital to efficient multi-robot interaction, but it is not yet clear how such communications are affected by the reality gap. For instance, multi-robot coordination is typically trained synchronously, but when deploying these policies to the real world, decentralised communication is asynchronous. Furthermore, randomness, such as message dropouts and delays, is typically not considered during synchronous training. There is a dearth of research that evaluates the robustness of models to such factors and their impact on the performance of policies. Decentralised mesh communication networks are required to operate MRS in the real world, which poses additional challenges to the sim-to-real transfer. This thesis presents a variety of different contributions and frameworks for deploying and testing multi-robot control and perception policies on decentralised real-world MRS. The first part of this thesis focuses on applying learnable communications to multi-robot coordination. We are the first to train a cooperative multi-agent coordination policy with learnable communication using Reinforcement learning (RL) through a differentiable communication channel and present simulation results for various multi-robot scenarios. To deploy these policies to the real world, we construct a fleet of nine agile ground-based robots, the Cambridge RoboMaster. We use this platform to conduct a physical ablation of multi-agent deployment methods, from centralised to decentralised, and demonstrate the deployment of different coordination policies, showing the effects of sim-to-real transfer. The second part focuses on learnable communications for multi-robot perception. Moving beyond coordination policies that rely on global state information from external localisation infrastructure, we explore training policies that utilise local sensor information from onboard monocular cameras. We train a GNN-based visual navigation policy, guiding a mobile robot to a target without requiring calibration. We then combine the insights on control and perception and introduce a cooperative, multi-robot visual spatial foundation model for decentralised perception and control for real-time real-world deployments. This model predicts relative poses and local spatial maps from monocular camera images and outperforms classical methods requiring overlapping camera views and predefined network infrastructures. We demonstrate the model's effectiveness by applying it to multi-robot formation control tasks, in which we show decentralised deployment in various indoor and outdoor scenes. This thesis bridges the gap between theoretical learning and practical deployment in MRS. The developed methods pave the way for more efficient and adaptable MRS capable of addressing real-world challenges.