Review of multimodal data and their applications for road maintenance

Abstract

The application of multimodal data in road maintenance has attracted considerable attention due to its potential to enhance decision-making processes and improve infrastructure resilience. This paper provides a comprehensive review of the utilisation of various modalities of multimodal data, including LiDAR, RGB images, thermal images, ground-penetrating radar (GPR), text, audio, and some others for road maintenance tasks. The research methodology thoroughly examines existing literature, categorising data modalities and analysing their respective applications. The paper discusses the integration and fusion of multimodal data, spatial and temporal analysis techniques, decision support systems, strategies for resilience and adaptability and information requirements in for road maintenance. It also explores data structures for integration into digital twin, advanced methodologies for sensor fusion, integration of new sensors and data types and multimodal sensors into road maintenance. This comprehensive review underscores the significance of multimodal data in enhancing the efficiency and effectiveness of road maintenance activities and identifies gaps in the automatic fusion of different modalities in the context of road asset management.