Automating graph-based geometric digital model generation for building digital twin applications from point cloud and image data

Abstract

Creating geometric digital twins of buildings remains a labor-intensive process, often limited to the reconstruction of structural elements. Non-structural components and their spatial relationships with indoor spaces are rarely integrated into a unified digital representation. This paper proposes a novel semi-automated method for generating graph-based geometric digital models for digital twins from 3D point cloud and image data. The approach extracts spatial and object information from these data based on deep learning methods. Multiple 2D detectors are trained on different public and customized datasets to broaden class coverage, and their predictions are mapped into a unified label space, fused per view, projected into 3D space, and merged into object instances that correspond to the same physical element. A new graph schema is then introduced to represent indoor spaces and elements, capturing both hierarchical and spatial relationships. The schema links each entity to its geometric representation and supports temporal snapshots. All extracted information is structured and stored in a graph database using the proposed schema. The method is validated on two real-world datasets, one residential house and one institutional facility, capturing a broader and more differentiated range of object classes across different building types and topologies. The promising results indicate that the method has the potential to be generalized to a wider range of buildings.