Automated Object Segmentation in Existing Industrial Facilities

Abstract

Shape segmentation from point cloud data is a core step of the digital twinning process for industrial facilities. However, this process is labour-intensive with 90% of the cost being spent on converting point cloud data to a model. This counteracts the perceived value of the resulting model in managing and retrofitting the facilities and motivates the use of automation to reduce this cost. In the US alone, unplanned factory shutdowns due to maintenance cost $50 billion per year. Better documenting the existing conditions can significantly circumvent irreversible damages and decrease the frequency of shutdowns, thus boosting the productivity of industrial assets. This explains why there is a huge market demand for less labour-intensive industrial documentation.

Shape segmentation in the literature has so far mostly focused on cylinders, with state-of- the-art methods achieving 60-70% precision and recall for cylinder detection. Such performance is promising, but far from drastically eliminating the manual labour cost, as all other shapes have to be segmented manually. Yet the search space is massive; industrial facilities contain thousands of object types, making automated detection an impossible problem. Hence, there is a direct need to prioritise the most tedious to model objects.

The objective of this PhD research is to devise, implement and benchmark a novel framework that can accurately generate individual labelled point clusters of the most important shapes of existing industrial facilities with minimal manual effort in a generic point-level format. This is addressed by first identifying the most important shapes to be modelled and then developing algorithms to efficiently detect those shapes. The former is achieved by answering the following three general research questions: a) what are the most frequent industrial object types?, b) what is the time to model the most frequent object types in state-of-the-art commercial software? and c) what is the performance of state-of-the-art tools in terms of automated object detection? The proposed methodology employs a statistical analysis to identify the most frequent industrial object types and then manually models those to estimate the average man-hours needed for each type. Then, it evaluates the state-of-the-art automated cylinder extraction tool and concludes a 64% reduction in manual modelling time of cylinders. This leads to focus on reducing the remaining man-hours for cylinder modelling as well as for manual modelling of the remaining industrial objects, which are still substantial. This is achieved by answering the following technical research questions: (1) how to automatically segment the most important industrial shapes from point cloud data with varying point densities and occlusions without relying on prior knowledge? (2) how to minimise the time for manually assigning class labels to points? and (3) how to automatically segment instance point clusters with less manual labour compared to the state-of-the-art?

The proposed framework employs a combination of deep learning and geometric methods to segment the points into classes and individual instances. Along the way, the author generates the largest to-date dataset of laser scanned industrial facilities used for training and evaluation. Experiments reveal that the method can work reliably in complex and incomplete point clouds of industrial facilities, yielding 82% class segmentation accuracy. Compared to the current state-of-practice, the proposed framework can realise estimated time-savings of 30% on average.

Contributions. This PhD research provides the unprecedented ability to rapidly and intelligently segment point clusters based on quantitative measurements. This is a huge leap over the current practice and a significant step towards the automated generation of industrial Digital Twins. As a result, the knowledge created in this PhD research will enable the future development of novel, automated applications for real-time factory maintenance, planned and unplanned downtime reduction.