Efficient Continual Learning and On-Device Training for Mobile and IoT Devices

Abstract

The surge in mobile phones, wearables, and Internet of Things (IoT) devices has resulted in an abundance of sensor data. This played a pivotal role in the widespread adoption of deep neural networks (DNN) to support various real-world scenarios in mobile computing, including personalising user experiences and enabling adaptive household robots. Such use cases require DNNs to continuously learn and adapt to changing real-world conditions, despite constraints such as limited labelled data, memory, and computational power. However, achieving continual learning (CL) and on-device training on resource-constrained edge devices poses significant challenges, both in terms of resource limitations and the complexity of learning algorithms to continually learn new tasks without forgetting old ones. This dissertation tackles these challenges by developing hardware-aware algorithms and systems that substantially optimise the utilisation of system resources for deployed DNNs on embedded and IoT platforms, while upholding high accuracy. Initially, this dissertation explores the feasibility and applicability of various CL methods in diverse mobile sensing applications, taking into account constraints such as low computational capability, limited memory and storage. Drawing from this analysis, we identify the bottlenecks of existing CL systems. We then overcome the stringent resource limitations of mobile and embedded systems by crafting a novel CL approach called FastICARL that optimises the computational and storage demands of the representative CL method. Subsequently, to seamlessly support on-device training and CL on extremely resource-constrained devices like microcontrollers (MCUs), we propose YONO, a multi-task inference system enabling in-memory model execution and seamless switching of varying tasks involving multiple user applications, which could facilitate on-device training and CL with multi-user scenarios. Furthermore, we propose TinyTrain, an efficient on-device training approach that minimises resource requirements while coping with limited data availability. TinyTrain significantly reduces memory usage, training latency, and energy consumption by effectively identifying and updating the essential model parts on the fly. This makes TinyTrain crucial for enabling CL on edge devices with limited resources. Finally, the dissertation pushes the boundaries of CL in mobile computing by extending CL to embedded systems and highly resource-constrained MCUs. Building on our thorough analysis of CL and the technology developed for resource-constrained devices, we propose LifeLearner, an efficient CL system that comprehensively addresses on-device resource requirements namely data, memory, and computation. LifeLearner is optimised for various hardware platforms such as edge devices (Jetson Nano and Raspberry Pi 3B+) and the STM32H747 MCU. Specifically, we co-design meta-learning with an efficient rehearsal strategy, enabling LifeLearner to rapidly learn new classes using only a few samples while alleviating forgetting. We then design a CL-tailored Compression Module that minimises the resource overheads of CL and hardware-aware optimisations to enhance overall runtime efficiency. The methodologies developed, systems optimised, and insights gleaned from this dissertation lay the foundation for the widespread deployment of continual and on-device training systems that dynamically adapt to users and environments while operating efficiently within resource-constrained settings.