Complex Instance Segmentation in Point Clouds with Images and 3D Models

Abstract

A geometric digital twin can help monitor the progress, control the quality, and simulate the energy of a building during its construction and operation stages. However, current methods cannot match and segment instances with a high-resolution result in real, complex environments regarding mechanical, electrical, and plumbing (MEP) objects, since the geometry of as-built MEP objects is complex and often deviates from as-designed models in terms of position, orientation, and scale. To this end, this paper proposes a hybrid method by fusing point clouds, images, and 3D models to efficiently segment complex MEP components in buildings. An experiment targeting heating terminals in a real three-floor staircase space shows the feasibility and practicality of the proposed method.